【Azure Developer】使用REST API获取Activity Logs、传入Data Lake的数据格式问题

简介: 【Azure Developer】使用REST API获取Activity Logs、传入Data Lake的数据格式问题

问题一:.  如何在用REST API获取活动日志时,控制输出的项?

【答】参考REST API对于获取活动日志的说明接口,在参数是$filter$select中可以分别控制过滤条件输出项

GET https://management.chinacloudapi.cn/subscriptions/{subscriptionId}/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-01&$filter={$filter}&$select={$select}

注:management.chinacloudapi.cn为中国区Azure的管理终结点,management.azure.com为全球Azure的管理终结点

URI Parameters

Name Description

$filter

Reduces the set of data collected.

This argument is required and it also requires at least the start date/time.

The $filter argument is very restricted and allows only the following patterns.

- List events for a resource group:

$filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceGroupName eq 'resourceGroupName'.

- List events for resource:

$filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceUri eq 'resourceURI'.

- List events for a subscription in a time range:

$filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z'.

- List events for a resource provider:

$filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and resourceProvider eq 'resourceProviderName'.

- List events for a correlation Id:

$filter=eventTimestamp ge '2014-07-16T04:36:37.6407898Z' and eventTimestamp le '2014-07-20T04:36:37.6407898Z' and correlationId eq 'correlationID'.


NOTE: No other syntax is allowed.

$select

Used to fetch events with only the given properties.

The $select argument is a comma separated list of property names to be returned.

Possible values are: 

authorization, claims, correlationId, description, eventDataId

eventName, eventTimestamp, httpRequest, level, operationId

operationName, properties, resourceGroupName, resourceProviderName

resourceId, status, submissionTimestamp, subStatus, subscriptionId

在Filter和select可以配置的每一项说明,请查看文末的附录一

 

问题二:是否可以往Data Lake中注入Json格式文件?

【答】根据官方文档示例,如果使用Databricks进行(SQL查询语句)进行数据分析,是支持Json格式的数据,示例中使用的Json格式如下图所示:

参考使用 Databricks 分析数据一文: https://docs.azure.cn/zh-cn/storage/blobs/data-lake-storage-quickstart-create-databricks-account#ingest-sample-data

 

问题三:如何用C#向Data Lake中插入数据文件?

【答】可以,需要先安装 Azure.Storage.Files.DataLake NuGet 包。然后添加如下引用

using Azure.Storage.Files.DataLake;

using Azure.Storage.Files.DataLake.Models;

using Azure.Storage;

using System.IO;

using Azure;

将文件上传到目录,则参考以下代码即可:

public async Task UploadFile(DataLakeFileSystemClient fileSystemClient)
{
    DataLakeDirectoryClient directoryClient =
        fileSystemClient.GetDirectoryClient("my-directory");
    DataLakeFileClient fileClient = await directoryClient.CreateFileAsync("uploaded-file.txt");
    FileStream fileStream = 
        File.OpenRead("C:\\file-to-upload.txt");
    long fileSize = fileStream.Length;
    await fileClient.AppendAsync(fileStream, offset: 0);
    await fileClient.FlushAsync(position: fileSize);
}

完整实例参考使用 .NET 管理 Azure Data Lake Storage Gen2https://docs.azure.cn/zh-cn/storage/blobs/data-lake-storage-directory-file-acl-dotnet#upload-a-file-to-a-directory

 

 

附录一:在和和Filter和select可配置的选择项的EventData属性

The Azure event log entries are of type EventData

Name Type Description
authorization

The sender authorization information.

caller
  • string

the email address of the user who has performed the operation,

the UPN claim or SPN claim based on availability.

category

the event category.

claims
  • object

key value pairs to identify ARM permissions.

correlationId
  • string

the correlation Id, usually a GUID in the string format.

The correlation Id is shared among the events that belong to the same uber operation.

description
  • string

the description of the event.

eventDataId
  • string

the event data Id. This is a unique identifier for an event.

eventName

the event name. This value should not be confused with OperationName.

For practical purposes, OperationName might be more appealing to end users.

eventTimestamp
  • string

the timestamp of when the event was generated by the Azure service processing the

request corresponding the event. It in ISO 8601 format.

httpRequest

the HTTP request info. Usually includes the 'clientRequestId', 'clientIpAddress'

(IP address of the user who initiated the event) and 'method' (HTTP method e.g. PUT).

id
  • string

the Id of this event as required by ARM for RBAC. It contains the EventDataID

and a timestamp information.

level

the event level

operationId
  • string

It is usually a GUID shared among the events corresponding to single operation.

This value should not be confused with EventName.

operationName

the operation name.

properties
  • object

the set of <Key, Value> pairs (usually a Dictionary<String, String>)

that includes details about the event.

resourceGroupName
  • string

the resource group name of the impacted resource.

resourceId
  • string

the resource uri that uniquely identifies the resource that caused this event.

resourceProviderName

the resource provider name of the impacted resource.

resourceType

the resource type

status

a string describing the status of the operation.

Some typical values are: Started, In progress, Succeeded, Failed, Resolved.

subStatus

the event sub status.

Most of the time, when included, this captures the HTTP status code of the REST call.

Common values are:

OK (HTTP Status Code: 200),

Created (HTTP Status Code: 201),

Accepted (HTTP Status Code: 202),

No Content (HTTP Status Code: 204),

Bad Request(HTTP Status Code: 400),

Not Found (HTTP Status Code: 404),

Conflict (HTTP Status Code: 409),

Internal Server Error (HTTP Status Code: 500),

Service Unavailable (HTTP Status Code:503),

Gateway Timeout (HTTP Status Code: 504)

submissionTimestamp
  • string

the timestamp of when the event became available for querying via this API.

It is in ISO 8601 format. This value should not be confused eventTimestamp.

As there might be a delay between the occurrence time of the event,

and the time that the event is submitted to the Azure logging infrastructure.

subscriptionId
  • string

the Azure subscription Id usually a GUID.

tenantId
  • string

the Azure tenant Id

 

相关文章
|
26天前
|
JSON API 数据格式
Amazon商品详情API,json数据格式示例参考
亚马逊商品详情API接口返回的JSON数据格式通常包含丰富的商品信息,以下是一个简化的JSON数据格式示例参考
|
1月前
|
缓存 API 网络架构
掌握现代API开发:GraphQL vs REST
【10月更文挑战第24天】本文深入探讨了现代API开发中两种主流技术——GraphQL和REST的设计理念、技术特点及实际开发中的对比分析。GraphQL通过声明式数据请求和强类型系统提供更高的灵活性和性能,而REST则以其无状态特性和成熟的生态系统见长。文章还讨论了两者在客户端-服务器交互、安全性和工具支持方面的优劣,帮助开发者根据项目需求做出明智选择。
|
1月前
|
JSON API 数据格式
店铺所有商品列表接口json数据格式示例(API接口)
当然,以下是一个示例的JSON数据格式,用于表示一个店铺所有商品列表的API接口响应
|
2月前
|
JSON API 数据安全/隐私保护
拍立淘按图搜索json数据格式示例(API接口)
拍立淘按图搜索API接口为电商平台和购物应用提供了强大的图像搜索功能,能够显著提升用户的购物体验和搜索效率。开发者可以根据自己的需求调用此接口,并处理返回的JSON格式数据来展示推荐商品
|
2月前
|
JSON JavaScript API
(API接口系列)商品详情数据封装接口json数据格式分析
在成长的路上,我们都是同行者。这篇关于商品详情API接口的文章,希望能帮助到您。期待与您继续分享更多API接口的知识,请记得关注Anzexi58哦!
|
3月前
|
JSON 中间件 API
开发REST API3-11
开发REST API3-11
|
3月前
|
JSON JavaScript API
编写REST API
编写REST API
64 2
|
2月前
|
Java API Maven
使用 Smart-doc 记录 Spring REST API
使用 Smart-doc 记录 Spring REST API
59 0
|
4月前
|
XML 安全 API
REST 和 SOAP API 有什么区别?
【8月更文挑战第31天】
231 0
|
4月前
|
JSON API 数据安全/隐私保护
哇塞!Django REST framework 太逆天啦!构建 API 服务从未如此轻松,你还不来试试?
【8月更文挑战第31天】Django REST framework(DRF)是基于Django框架的高效Web API开发工具,提供序列化、视图集、路由等功能,简化API构建流程。使用DRF可轻松实现数据的序列化与反序列化,并支持权限管理和认证机制以保障API安全。安装DRF只需通过`pip install djangorestframework`命令。要创建基本项目,先安装Django并创建新应用,定义模型、序列化器及视图集,最后配置路由。测试API时,可通过Postman发送HTTP请求验证功能。无论项目大小,DRF均能提供强大支持。
47 0