Data_Engineering_Part6
Page 1: HTTP APIs for Microsoft Fabric Configuration
HTTP API Requests
Get Staging Libraries:
GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/librariesGet Spark Compute:
GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/sparkcompute
Publishing the Environment
The changes made for staging libraries and Spark compute are cached and require publishing to take effect.
Sample Request to Trigger Publish:
HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/publish
Sample Response After Publishing
{
"publishDetails": {
"state": "Running",
"targetVersion": "46838a80-5450-4414-bea0-40fb6f3e0c0d",
"startTime": "2024-03-29T14:17:09.0697022Z",
"componentPublishInfo": {
"sparkLibraries": {
"state": "Running"
},
"sparkSettings": {
"state": "Running"
}
}
}
}Canceling a Publish
If needed, you can call the following API to cancel the publishing:
HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/cancelPublish
Page 2: Related Actions and Feedback
Related Content
Create, configure, and use an environment in Microsoft Fabric.
Item APIs - Environment feedback options are provided.
Was this page helpful?
[Yes] [No] Provide product feedback
Ask the community.
Page 3: Managing Resources in Fabric Environment
Resources Overview
Resources in Fabric environments provide a file system to manage files and folders, accessible from notebooks.
Collaboration is enabled when notebooks are attached to the same environment.
Operations on Resources
Common actions include creating folders, uploading files, and managing their properties.
Uploaded files will appear in notebooks when executed in this environment.
The storage has limits: maximum file size is 500MB, individual files are 50MB, and a total of 100 files and folders combined.
User Interface
Environment resources can be managed through both environment and notebook UI. Drag-and-drop functionality is supported for code snippets from the resources to the notebook code cells.
HTTP APIs for Microsoft Fabric Configuration
HTTP API Requests
Get Staging Libraries
Request: GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/libraries
Description: This API retrieves the list of staging libraries for a specific environment within Microsoft Fabric, which are essential for managing dependencies and ensuring the correct libraries are utilized during development.
Get Spark Compute
Request: GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/sparkcompute
Description: This request fetches the Spark compute configuration for the specified environment. It provides insights into current compute resources utilized for data processing operations.
Publishing the Environment
Important Note: Changes made to staging libraries and Spark compute settings are cached and require a publishing action to take effect in the live environment. This step is crucial to ensure that all modifications are applied and functional.
Sample Request to Trigger Publish
API Call: HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/publish
Purpose: Initiates the publishing process for staging changes, which applies the modifications to the target environment.
Sample Response After Publishing
{
"publishDetails": {
"state": "Running",
"targetVersion": "46838a80-5450-4414-bea0-40fb6f3e0c0d",
"startTime": "2024-03-29T14:17:09.0697022Z",
"componentPublishInfo": {
"sparkLibraries": {
"state": "Running"
},
"sparkSettings": {
"state": "Running"
}
}
}
}Explanation: The response indicates the state of the publishing process, including the target version and the start time of the operation. It also provides the status of individual components being published.
Canceling a Publish
Cancellation API: If needed, you can call the following API to cancel the ongoing publishing process:HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/cancelPublish
Use Case: This is useful for halting a publish if errors are detected early in the process or if the changes no longer need to be applied.
Related Actions and Feedback
Related Content
Detailed guides on creating, configuring, and using an environment in Microsoft Fabric to ensure optimal setups for data projects.
Item APIs provide various environment feedback options, making it easier for users to report issues or provide suggestions.
User Engagement
Was this page helpful? [Yes] [No]
Users can provide product feedback directly through the platform and engage with the community for insights and shared experiences.
Managing Resources in Fabric Environment
Resources Overview
File Management: Resources in Fabric environments provide an organized file system to manage files and folders, which are accessible from notebooks. This organizational structure is vital for maintaining efficient workflows.
Collaboration: Collaboration is enhanced as notebooks attached to the same environment can access shared resources seamlessly, promoting teamwork and data sharing.
Operations on Resources
Common Actions: Users can perform various operations on resources, including:
Creating folders for organization.
Uploading files needed for projects.
Managing properties such as permissions and metadata for effective control.
Storage Limits: It is essential to be aware of the storage limitations:
Maximum file size is capped at 500MB.
Individual files cannot exceed 50MB.
The maximum combined number of files and folders allowed is 100.
User Interface
The environment resources can be effectively managed through both the environment UI and notebook UI, which supports drag-and-drop functionality for code snippets from resources to notebook code cells. This feature simplifies the process of incorporating existing resources into new analyses and