Data_Engineering_Part6

Page 1: HTTP APIs for Microsoft Fabric Configuration

HTTP API Requests

• Get Staging Libraries:GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/libraries

• Get Spark Compute:GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/sparkcompute

Publishing the Environment

• The changes made for staging libraries and Spark compute are cached and require publishing to take effect.

• Sample Request to Trigger Publish:HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/publish

Sample Response After Publishing

{
  "publishDetails": {
    "state": "Running",
    "targetVersion": "46838a80-5450-4414-bea0-40fb6f3e0c0d",
    "startTime": "2024-03-29T14:17:09.0697022Z",
    "componentPublishInfo": {
      "sparkLibraries": {
        "state": "Running"
      },
      "sparkSettings": {
        "state": "Running"
      }
    }
  }
}

Canceling a Publish

• If needed, you can call the following API to cancel the publishing:HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/cancelPublish

Page 2: Related Actions and Feedback

Related Content

• Create, configure, and use an environment in Microsoft Fabric.

• Item APIs - Environment feedback options are provided.

• Was this page helpful?

  • [Yes] [No] Provide product feedback

  • Ask the community.

Page 3: Managing Resources in Fabric Environment

Resources Overview

• Resources in Fabric environments provide a file system to manage files and folders, accessible from notebooks.

• Collaboration is enabled when notebooks are attached to the same environment.

Operations on Resources

• Common actions include creating folders, uploading files, and managing their properties.

• Uploaded files will appear in notebooks when executed in this environment.

• The storage has limits: maximum file size is 500MB, individual files are 50MB, and a total of 100 files and folders combined.

User Interface

• Environment resources can be managed through both environment and notebook UI. Drag-and-drop functionality is supported for code snippets from the resources to the notebook code cells.

HTTP APIs for Microsoft Fabric Configuration

HTTP API Requests

Get Staging Libraries

• Request: GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/libraries

• Description: This API retrieves the list of staging libraries for a specific environment within Microsoft Fabric, which are essential for managing dependencies and ensuring the correct libraries are utilized during development.

Get Spark Compute

• Request: GET https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/sparkcompute

• Description: This request fetches the Spark compute configuration for the specified environment. It provides insights into current compute resources utilized for data processing operations.

Publishing the Environment

• Important Note: Changes made to staging libraries and Spark compute settings are cached and require a publishing action to take effect in the live environment. This step is crucial to ensure that all modifications are applied and functional.

Sample Request to Trigger Publish

• API Call: HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/publish

• Purpose: Initiates the publishing process for staging changes, which applies the modifications to the target environment.

Sample Response After Publishing

{
  "publishDetails": {
    "state": "Running",
    "targetVersion": "46838a80-5450-4414-bea0-40fb6f3e0c0d",
    "startTime": "2024-03-29T14:17:09.0697022Z",
    "componentPublishInfo": {
      "sparkLibraries": {
        "state": "Running"
      },
      "sparkSettings": {
        "state": "Running"
      }
    }
  }
}

• Explanation: The response indicates the state of the publishing process, including the target version and the start time of the operation. It also provides the status of individual components being published.

Canceling a Publish

• Cancellation API: If needed, you can call the following API to cancel the ongoing publishing process:HTTP POST https://api.fabric.microsoft.com/v1/workspaces/{{WORKSPACE_ID}}/environments/{{ARTIFACT_ID}}/staging/cancelPublish

• Use Case: This is useful for halting a publish if errors are detected early in the process or if the changes no longer need to be applied.

Related Actions and Feedback

Related Content

• Detailed guides on creating, configuring, and using an environment in Microsoft Fabric to ensure optimal setups for data projects.

• Item APIs provide various environment feedback options, making it easier for users to report issues or provide suggestions.

User Engagement

• Was this page helpful? [Yes] [No]

• Users can provide product feedback directly through the platform and engage with the community for insights and shared experiences.

Managing Resources in Fabric Environment

Resources Overview

• File Management: Resources in Fabric environments provide an organized file system to manage files and folders, which are accessible from notebooks. This organizational structure is vital for maintaining efficient workflows.

• Collaboration: Collaboration is enhanced as notebooks attached to the same environment can access shared resources seamlessly, promoting teamwork and data sharing.

Operations on Resources

• Common Actions: Users can perform various operations on resources, including:

  • Creating folders for organization.

  • Uploading files needed for projects.

  • Managing properties such as permissions and metadata for effective control.

• Storage Limits: It is essential to be aware of the storage limitations:

  • Maximum file size is capped at 500MB.

  • Individual files cannot exceed 50MB.

  • The maximum combined number of files and folders allowed is 100.

User Interface

• The environment resources can be effectively managed through both the environment UI and notebook UI, which supports drag-and-drop functionality for code snippets from resources to notebook code cells. This feature simplifies the process of incorporating existing resources into new analyses and