Skip to main content

NestJS API File Operations Using Azure Blob Storage

In this article, we are going to use Azure Blob Storage for all file operations like uploading, downloading, and deleting files from the NestJS API endpoint.

Azure Blob Storage:

Azure blob storage is Microsoft cloud storage. Blob storage can store a massive amount of file data as unstructured data. The unstructured data means not belong to any specific type, which means text or binary data. So something like images or pdf or videos to store in the cloud, then the most recommended is to use the blob store.

The key component to creating azure blob storage resource:

Storage Account:-
A Storage account gives a unique namespace in Azure for all the data we will save. Every object that we store in Azure Storage has an address. The address is nothing but the unique name of our Storage Account name. The combination of the account name and the Azure Storage blob endpoint forms the base address for each object in our Storage account. For example, if our Storage Account is named as 'myazurestorageaccount' then the base address will be like 'https://myazurestorageaccount.blob.core.windows.net'.

Containers:-
The containers are like folders in the file system. So storage account can have an unlimited number of containers. Inside each container can have an unlimited number of blobs.

Blobs:- 
Azure gives 3 types of blobs:
  • Block Blobs - Block of data managed individually. Most recommended for file uploading.
  • Append Blobs - Contains Block of data but with append operation.
  • Page Blogs

Create Azure Blob Storage In Azure Portal:

To consume any azure service we have to signup for the Azure portal, an additional bonus of the azure portal is users can use most of the service with free subscriptions. The free subscription is very handy for developers to learn about the azure service.

On the Azure portal dashboard page click on 'Create a resource.

Search for 'Storage Account' and select to create.

Fill 'Create Storage Account' form.
  • 'Resource group' - enter the name of the resource group. Resource groups are just separators or wrappers.
  • 'Storage account name' - enter the unique name and this name will be used for creating the base address.
  • 'Location - you can select the default or choose your nearest location value.
  • 'Performance' - chose your preferred option(for free subscription go for the 'Standard' option)
  • 'Account Kind' - 'StorageV2(general purpose v2)' is the recommended option.
  • 'Replication' - chose your preferred option or select the default option.
Then remaining tabs you can skip or chose for creating the Storage Account.

Once storage created. In the left side menu under 'Blob Service' select 'Containers' then click on Create new container which opens a form for creation.
Inside of the 'New Container' form, enter your Container name and then select option 'access level' for the container.

Now while creating the container I'm assigning access level 'anonymous read access for container and blob' which gives permission to access the blobs with the domain address directly.


Create NestJS Application:

Now our goal is to create NestJs endpoints for operating files in Azure Blob Storage. So let's create a NestJS application.
Command To Install CLI:
npm i -g @nestjs/cli
Command To Create NestJS App:
nest new your_project_name

Install Azure Blob Storage NodJS Package:

npm install @azure/storage-blob

Install NodeJS Multer Package:

In NestJS for handling file upload operation, it is the best choice to install the 'Multer' package. It is very effective in reading 'multipart/formdata'. The'Multer' also provides more type safety.
Multer Package:
npm i -D @types/multer

Azure Connectionstring:

There different ways like Authentication, ConnectionString, SharedSASToken, etc to communicate with the Azure Blob Storage.

In this demo, I'm going to use the ConnectionString in NestJS API to interact with the Azure Blob Storage. In Azure Storage Account, the left-hand side under settings select 'Access Keys' menu and it will display 'Key1' and 'Key2' access keys. We can choose any one of them.

NestJS Endpoint For Uploading File To Azure Blob Storage:

We have to implement our logic to create a file upload endpoint in NestJS.
src/app.service.ts:
import { BlobServiceClient, BlockBlobClient } from '@azure/storage-blob';
import { Injectable } from '@nestjs/common';


@Injectable()
export class AppService {
  azureConnection = "DefaultEndpointsProtocol=https;AccountName=naveenazurestore;AccountKey=NLNxbk2ElnaMaUucTLt1zQDU1eOznjYUmCaiIeJB6O+934iedXSmss+r5gDePmZE6y99X8zZ8AItA==;EndpointSuffix=core.windows.net";
  containerName = "upload-file";
  

  getBlobClient(imageName:string):BlockBlobClient{
    const blobClientService = BlobServiceClient.fromConnectionString(this.azureConnection);
    const containerClient = blobClientService.getContainerClient(this.containerName);
    const blobClient = containerClient.getBlockBlobClient(imageName);
    return blobClient;
  }

  async upload(file:Express.Multer.File){
    const blobClient = this.getBlobClient(file.originalname);
    await blobClient.uploadData(file.buffer);
  }
}
  • (Line: 1) Instances like 'BlobServiceClient' & 'BlockBlobClient' loading from '@azure/storage-blob'.
  • (Line: 7) Azure connection string.
  • (Line: 8) Azure Blob Storage container name. This our target container to upload our files from our NestJS API endpoint.
  • (Line: 11-16) The 'getBlobClient' method returns 'BlockBlobClient' from our specified 'Blob'(here Blob means our file to upload).
  • (Line: 12)  Configuring our azure connection string into the 'BlobServiceClient'.
  • (Line: 13)  Configuring our container name.
  • (Line: 14)  Configuring the name of our file to be upload and returning the 'BlockBlobClient' instance. The 'BlockBlobClient' now contains all the configurations like connection, container, filename.
  • (Line: 18-21) The 'upload' method contains logic to upload files to Azure Blob Storage. Its input parameter is a file of type 'Multer.File'.
  • (Line: 19) Fetching 'BlockBlobClient' instance.
  • (Line: 20) Invoking file upload call by using 'uploadData' method from the 'BlockBlobClient' instance. The 'uploadData' takes file buffer as input.
src/app.controller.cs:
import { Controller, Post, UploadedFile, UseInterceptors } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { AppService } from './app.service';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}


  @Post('upload')
  @UseInterceptors(FileInterceptor('myfile'))
  async upload(@UploadedFile() file: Express.Multer.File):Promise<string>{
    await this.appService.upload(file);
    return "uploaded";
  }
}
  • (Line: 11) The 'FileInterceptor' reads all file stream from the 'myfile'(name attribute value or form data value).
  • (Line: 12) The '@UploadedFile()' decorator grabs the all file information and assigns it to the variable like 'file' in our sample. Here we are defining the type as 'Express.Multer.File'.
  • (Line: 13) Passing file to the 'upload' method.
Now let's test our endpoint from the 'Postman' tool(Note: While uploading from Postman tool, clear all headers).
Now check in the Azure dashboard.

Create NestJS Endpoint To Read File:

The new endpoint going to fetch the steam of file from the Azure Blob Storage and then render it on the browser.
src/app.service.ts:
async getfileStream(fileName: string){
  const blobClient = this.getBlobClient(fileName);
  var blobDownloaded = await blobClient.download();
  return blobDownloaded.readableStreamBody;
}
  • Fetching required image or file stream from Azure Blob Storage by using the 'download()' method of 'BlockBlobClien' instance.
src/app.controller:
import { Controller, Get, Header, Post, Query, Res, UploadedFile, UseInterceptors } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { AppService } from './app.service';
// some code hidden for display purpose
@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}


  @Get('read-image')
  @Header('Content-Type','image/webp')
  async readImage(@Res() res,@Query('filename') filename){
    const file = await this.appService.getfileStream(filename);
    return file.pipe(res);
  }
}
  • (Line: 11) '@Header()' decorator used to specify the 'Content-Type' and its value 'image/webp'(webp is image format like jpg, png, webp, etc). Because of this header value, the image stream going to be rendered on the browser.
  • (Line: 12) '@Res()' decorator is response object. '@Query()' decorator to capture query params.
  • (Line: 13-14) After getting the file stream from Azure Blob Storage, we are flushing the stream into the response object.
Now try to access this new endpoint and check the result.

Create NestJS Endpoint To Download Image:

The new endpoint on opening it on a browser it's going to download the image from the Azure blob storage.

So logic to download image same as the read image endpoint just we created above, the only difference we need to add a one addition header decorator here.
src/app.controller.cs:
@Get('download-image')
@Header('Content-Type','image/webp')
@Header('Content-Disposition', 'attachment; filename=test.webp')
async downloadImage(@Res() res,@Query('filename') filename){
    const file = await this.appService.getfileStream(filename);
    return file.pipe(res);
}
(Line: 3) This header 'Content-Disposition' and its value 'attachment' makes the browser understand that response should be downloaded.

Create NestJS Endpoint To Delete Image:

The new endpoint going to delete the files from the Azure blob storage.
src/app.service.ts:
async delete(filename: string){
 const blobClient = this.getBlobClient(filename);
 await blobClient.deleteIfExists();
}
  • The 'deleteIfExists' method going to delete the file if exists on the Azure blob storage.
src/app.controller.ts:
@Get('delete-image')
async delete(@Query('filename') filename){
 await this.appService.delete(filename);
 return "deleted";
}
That's all about file operation in Azure Blob Storage from the NestJS application.

Video Session:

Support Me!
Buy Me A Coffee PayPal Me

Wrapping Up:

Hopefully, I think this article delivered some useful information about NestJS API file operations using Azure Blob Storage. I love to have your feedback, suggestions, and better techniques in the comment section below.

Refer:

Follow Me:

Comments

  1. Nice tutorial. I was looking for a file upload example using NestJS and possibly Multer... This video gave me some good hints...

    ReplyDelete
  2. This was really helpful. Thanks alot

    ReplyDelete

Post a Comment

Popular posts from this blog

.NET6 Web API CRUD Operation With Entity Framework Core

In this article, we are going to do a small demo on AspNetCore 6 Web API CRUD operations. What Is Web API: Web API is a framework for building HTTP services that can be accessed from any client like browser, mobile devices, desktop apps. In simple terminology API(Application Programming Interface) means an interface module that contains a programming function that can be requested via HTTP calls to save or fetch the data for their respective clients. Some of the key characteristics of API: Supports HTTP verbs like 'GET', 'POST', 'PUT', 'DELETE', etc. Supports default responses like 'XML' and 'JSON'. Also can define custom responses. Supports self-hosting or individual hosting, so that all different kinds of apps can consume it. Authentication and Authorization are easy to implement. The ideal platform to build REST full services. Create A .NET6 Web API Application: Let's create a .Net6 Web API sample application to accomplish our

A Small Guide On NestJS Queues

NestJS Application Queues helps to deal with application scaling and performance challenges. When To Use Queues?: API request that mostly involves in time taking operations like CPU bound operation, doing them synchronously which will result in thread blocking. So to avoid these issues, it is an appropriate way to make the CPU-bound operation separate background job.  In nestjs one of the best solutions for these kinds of tasks is to implement the Queues. For queueing mechanism in the nestjs application most recommended library is '@nestjs/bull'(Bull is nodejs queue library). The 'Bull' depends on Redis cache for data storage like a job. So in this queueing technique, we will create services like 'Producer' and 'Consumer'. The 'Producer' is used to push our jobs into the Redis stores. The consumer will read those jobs(eg: CPU Bound Operations) and process them. So by using this queues technique user requests processed very fastly because actually

Part-1 Angular JWT Authentication Using HTTP Only Cookie[Angular V13]

In this article, we are going to implement a sample angular application authentication using HTTP only cookie that contains a JWT token. HTTP Only JWT Cookie: In a SPA(Single Page Application) Authentication JWT token either can be stored in browser 'LocalStorage' or in 'Cookie'. Storing JWT token inside of the cookie then the cookie should be HTTP Only. The HTTP-Only cookie nature is that it will be only accessible by the server application. Client apps like javascript-based apps can't access the HTTP-Only cookie. So if we use authentication with HTTP only JWT cookie then we no need to implement custom logic like adding authorization header or storing token data, etc at our client application. Because once the user authenticated cookie will be automatically sent to the server by the browser on every API call. Authentication API: To implement JWT cookie authentication we need to set up an API. For that, I had created a mock authentication API(Using the NestJS Se

Usage Of CancellationToken In Asp.Net Core Applications

When To Use CancellationToken?: In a web application request abortion or orphan, requests are quite common. On users disconnected by network interruption or navigating between multiple pages before proper response or closing of the browser, tabs make the request aborted or orphan. An orphan request can't deliver a response to the client, but it will execute all steps(like database calls, HTTP calls, etc) at the server. Complete execution of an orphan request at the server might not be a problem generally if at all requests need to work on time taking a job at the server in those cases might be nice to terminate the execution immediately. So CancellationToken can be used to terminate a request execution at the server immediately once the request is aborted or orphan. Here we are going to see some sample code snippets about implementing a CancellationToken for Entity FrameworkCore, Dapper ORM, and HttpClient calls in Asp.NetCore MVC application. Note: The sample codes I will show in

Unit Testing Asp.NetCore Web API Using xUnit[.NET6]

In this article, we are going to write test cases to an Asp.NetCore Web API(.NET6) application using the xUnit. xUnit For .NET: The xUnit for .Net is a free, open-source, community-focused unit testing tool for .NET applications. By default .Net also provides a xUnit project template to implement test cases. Unit test cases build upon the 'AAA' formula that means 'Arrange', 'Act' and 'Assert' Arrange - Declaring variables, objects, instantiating mocks, etc. Act - Calling or invoking the method that needs to be tested. Assert - The assert ensures that code behaves as expected means yielding expected output. Create An API And Unit Test Projects: Let's create a .Net6 Web API and xUnit sample applications to accomplish our demo. We can use either Visual Studio 2022 or Visual Studio Code(using .NET CLI commands) to create any.Net6 application. For this demo, I'm using the 'Visual Studio Code'(using the .NET CLI command) editor. Create a fo

Blazor WebAssembly Custom Authentication From Scratch

In this article, we are going to explore and implement custom authentication from the scratch. In this sample, we will use JWT authentication for user authentication. Main Building Blocks Of Blazor WebAssembly Authentication: The core concepts of blazor webassembly authentication are: AuthenticationStateProvider Service AuthorizeView Component Task<AuthenticationState> Cascading Property CascadingAuthenticationState Component AuthorizeRouteView Component AuthenticationStateProvider Service - this provider holds the authentication information about the login user. The 'GetAuthenticationStateAsync()' method in the Authentication state provider returns user AuthenticationState. The 'NotifyAuthenticationStateChaged()' to notify the latest user information within the components which using this AuthenticationStateProvider. AuthorizeView Component - displays different content depending on the user authorization state. This component uses the AuthenticationStateProvider

Angular 14 Reactive Forms Example

In this article, we will explore the Angular(14) reactive forms with an example. Reactive Forms: Angular reactive forms support model-driven techniques to handle the form's input values. The reactive forms state is immutable, any form filed change creates a new state for the form. Reactive forms are built around observable streams, where form inputs and values are provided as streams of input values, which can be accessed synchronously. Some key notations that involve in reactive forms are like: FormControl - each input element in the form is 'FormControl'. The 'FormControl' tracks the value and validation status of form fields. FormGroup - Track the value and validate the state of the group of 'FormControl'. FormBuilder - Angular service which can be used to create the 'FormGroup' or FormControl instance quickly. Form Array - That can hold infinite form control, this helps to create dynamic forms. Create An Angular(14) Application: Let'

How Response Caching Works In Asp.Net Core

What Is Response Caching?: Response Caching means storing of response output and using stored response until it's under it's the expiration time. Response Caching approach cuts down some requests to the server and also reduces some workload on the server. Response Caching Headers: Response Caching carried out by the few Http based headers information between client and server. Main Response Caching Headers are like below Cache-Control Pragma Vary Cache-Control Header: Cache-Control header is the main header type for the response caching. Cache-Control will be decorated with the following directives. public - this directive indicates any cache may store the response. private - this directive allows to store response with respect to a single user and can't be stored with shared cache stores. max-age - this directive represents a time to hold a response in the cache. no-cache - this directive represents no storing of response and always fetch the fr

Different HttpClient Techniques To Consume API Calls In Minimal API[.NET6]

In this article, we are going to implement different HttpClient techniques to consume API calls in minimal API. The different HttpClient techniques that we are going to explore are like: Register HttpClient Object Explicitly In DI(Dependency Injection Service) Named Client Type Client HttpRequestMessage Object Create A .NET6 Minimal API Project: Let's create a .Net6 Minimal API sample project to accomplish our demo. We can use either Visual Studio 2022 or Visual Studio Code(using .NET CLI commands) to create any.Net6 application. For this demo, I'm using the 'Visual Studio Code'(using the .NET CLI command) editor. CLI command For Minimal API Project dotnet new webapi -minimal -o Your_Project_Name Create A Third Party API Response Model: Here I'm going to use a free third-party rest API that is "https://jsonplaceholder.typicode.com/posts". So to receive the response let's create a response model like 'Post.cs'. Program.cs:(Add Post.cs c

.Net5 Web API Managing Files Using Azure Blob Storage

In this article, we are going to understand the different file operations like uploading, reading, downloading, and deleting in .Net5 Web API application using Azure Blob Storage. Azure Blob Storage: Azure blob storage is Microsoft cloud storage. Blob storage can store a massive amount of file data as unstructured data. The unstructured data means not belong to any specific type, which means text or binary data. So something like images or pdf or videos to store in the cloud, then the most recommended is to use the blob store. The key component to creating azure blob storage resource: Storage Account:- A Storage account gives a unique namespace in Azure for all the data we will save. Every object that we store in Azure Storage has an address. The address is nothing but the unique name of our Storage Account name. The combination of the account name and the Azure Storage blob endpoint forms the base address for each object in our Storage account. For example, if our Storage Account is n