Specifically, we’ll cover:

• A brief introduction to Apache Kafka and its core concepts.
• Setting up a local Kafka cluster using Docker.
• Running Kafka UI to inspect brokers, topics, and messages.
• Creating a topic
• Creating a Kafka Producer in .NET.
• Creating a Kafka Consumer in .NET.
• Publishing and consuming movie events.

By the end of this article, you’ll have a working Kafka environment running locally and a basic understanding of how Kafka applications are built with .NET.

I will use the following tools:

• Visual Studio 2026
• .NET 10
• Docker Desktop
• Git

And the following technologies:

• Apache Kafka
• Kafka UI by Provectus
• Confluent.Kafka (.NET client library)
• Docker Compose

Apache Kafka

Apache Kafka has become one of the most widely adopted event streaming platforms for building scalable, reliable, and event-driven applications. It enables applications to exchange data asynchronously, allowing producers and consumers to operate independently while handling millions of events per second.

Although Kafka is widely used in distributed systems and microservices, getting started can seem overwhelming due to the number of concepts involved, such as brokers, topics, partitions, offsets, consumer groups, and replication.

So, let's start with understanding the key concepts.

Kafka Key Concepts

The following diagram illustrates the main Kafka components and how they interact.

Producer

A Producer is an application that publishes events to Kafka.

Unlike traditional request-response communication, producers do not send messages directly to consumers. Instead, they publish events to Kafka, allowing multiple independent consumers to process those events whenever they become available.

Consumer

A Consumer is an application that subscribes to one or more Kafka topics and continuously reads events from them.

Multiple consumers can read the same topic independently. Alternatively, consumers can work together as part of a Consumer Group, allowing the processing workload to be distributed across multiple application instances.

Topic

A Topic is a named stream of events. You can think of it as a durable log where related events are stored.

Kafka topics have several important characteristics:

• A topic is a durable log of events.
• Topics are append-only, meaning new messages are always written to the end of the log.
• Messages stored in a topic are immutable once written.
• The retention period is configurable, allowing messages to be retained for hours, days, weeks, or even indefinitely.

One important thing to remember is that consuming a message does not remove it from the topic. Messages remain available until their configured retention period expires.

Event (Message)

An Event (also called a Message or Record) represents a single piece of information stored in Kafka.

Every Kafka message consists of four parts:

Timestamp

The time when the event was created or appended to Kafka.

Key

Used to determine which partition stores the message. The key does not have to be unique.

Value

The actual payload of the event.

Headers

Optional key-value pairs containing metadata such as correlation IDs, tracing information, or custom application data.

Broker

A Broker is a computer, virtual machine, cloud instance, or Docker container running the Kafka broker process.

A Kafka cluster typically consists of multiple brokers working together.

Each broker is responsible for:

• Managing the partitions stored on the broker.
• Handling write requests from producers.
• Handling read requests from consumers.
• Managing replication of partitions for fault tolerance.

To ensure high availability, each partition is replicated across multiple brokers.

For every partition:

• One replica acts as the Leader.
• The remaining replicas act as Followers.

Under normal operation:

• Producers write to the Leader.
• Consumers read from the Leader.
• Followers continuously replicate data from the Leader.

If the Leader becomes unavailable, Kafka automatically promotes one of the synchronized Followers to become the new Leader.

This leader-follower model is one of Kafka’s key mechanisms for providing fault tolerance and high availability.

The Controller is a special Kafka broker responsible for managing the Kafka cluster itself. It does not store application data or handle producer and consumer requests. Instead, it manages the cluster’s metadata and coordinates administrative operations.

Consumer Group

A Consumer Group is a collection of consumers working together to process events from a topic.

Kafka distributes partitions among consumers within the same group so that each partition is processed by only one consumer.

As more consumers are added to the group, Kafka automatically rebalances partition ownership, allowing applications to scale horizontally.

Control Plane (ZooKeeper / KRaft)

Kafka clusters require a control plane responsible for managing cluster metadata and coordinating brokers.

Historically, Kafka relied on Apache ZooKeeper for these tasks. Modern Kafka versions introduce KRaft (Kafka Raft Metadata mode), which replaces ZooKeeper with a built-in metadata quorum.

Regardless of which option is used, the responsibilities are the same:

• Managing broker membership.
• Tracking the Leader for every partition.
• Storing topic and partition metadata.
• Coordinating controller elections.
• Managing overall cluster metadata.

New Kafka deployments are recommended to use KRaft, while many existing production environments still rely on ZooKeeper.

Advantages of Kafka

• Kafka stores messages for a configurable retention period, allowing consumers to read messages at any time within that window.
• Kafka consumers can replay messages from any offset, making it easy to reprocess data.
• Kafka supports consumer groups for load balancing and parallel processing.
• Kafka provides message ordering within partitions.
• Kafka can be run on-premises or in any cloud.
• Kafka integrates with Kafka Streams and ksqlDB for real-time stream processing.

Now the fun part starts😀

We will set up our local Kafka and Kafka UI and then will create a producer application which sends messages to the movies topic and a consumer application that reads messages from this topic.

Setting up Local Kafka and Kafka UI

Create a docker-compose.yml with the following contents:

services:
  kafka:
    image: apache/kafka:latest
    container_name: kafka
    hostname: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: "broker,controller"
      KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka:29093"
      KAFKA_LISTENERS: "CONTROLLER://:29093,PLAINTEXT_HOST://:9092,PLAINTEXT://:19092"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT_HOST://localhost:9092,PLAINTEXT://kafka:19092"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "CONTROLLER:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,PLAINTEXT:PLAINTEXT"
      KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER"
      KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_SHARE_COORDINATOR_STATE_TOPIC_MIN_ISR: 1
      KAFKA_LOG_DIRS: "/tmp/kraft-combined-logs"
      CLUSTER_ID: "4L6g3nShT-eMCtK--X86sw"

  kafka-ui:
    image: provectuslabs/kafka-ui:latest
    container_name: kafka-ui
    depends_on:
      - kafka
    ports:
      - "8080:8080"
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:19092

Then run the following command:

docker compose up -d

Now you will see 2 running containers in your Docker Desktop.

• kafka
• kafka-ui

We are using UI for Apache Kafka tool and you can open it from Docker Desktop using the kafka-ui container.

Now, let’s create the movies topic from the Kafka UI.

Go to Topics and select Add a Topic. Then fill the values as you want and click Create topic.

We can see the created topic as below:

Creating a Producer Application

Now that our Kafka cluster is running and the movies topic has been created, it is time to build the first .NET application: the Producer.

The Producer application will create movie events and publish them to Kafka. In our example, the producer will send movie data to the movies topic.

Open Visual Studio and create a new Console App project. Name it something like KafkaProducer.

After the project is created, click Dependencies -> Manage Nuget Packages… and add Confluent.Kafka package.

The Confluent.Kafka package is the official and most commonly used Kafka client for .NET applications.

We will need to have some configuration related to our Kafka instance. Create Configuration folder under the project and add KafkaConfiguration class there with the following code:

namespace KafkaProducer.Configuration;

public class KafkaConfiguration
{
    public string BootstrapServers { get; set; } = string.Empty;
}

Lets use appsettings.json to store our Kafka configuration values. Create this file under the project and add the address of the Kafka container in your Docker Desktop.

{
  "KafkaConfiguration": {
    "BootstrapServers": "localhost:9092"
  }
}

Before publishing events to Kafka, we need a simple model to represent a movie.

Create Models folder under the project and then add a file named Movie.cs there with the following code:

namespace KafkaProducer.Models;

public class Movie
{
    public long Id { get; set; }
    public string Title { get; set; } = string.Empty;
    public string Director { get; set; } = string.Empty;
    public int ReleaseYear { get; set; }
    public string Genre { get; set; } = string.Empty;
    public double Rating { get; set; }
}

Now let’s add the ProducerService to our application. Create Services folder under the project and add the ProducerService there with the following code.

using Confluent.Kafka;
using KafkaProducer.Configuration;

namespace KafkaProducer.Services;

public class ProducerService<T> where T : class
{
    private readonly IProducer<string, T> _producer;

    public ProducerService(KafkaConfiguration kafkaConfiguration)
    {
        var producerConfig = new ProducerConfig
        {
            BootstrapServers = kafkaConfiguration.BootstrapServers,
            Acks = Acks.All
        };

        _producer = new ProducerBuilder<string, T>(producerConfig)
            .SetValueSerializer(new JsonSerializer<T>())
            .Build();
    }

    public async Task ProduceMessageAsync(string topic, string key, T value, IEnumerable<Header>? headers = null)
    {
        try
        {
            var message = new Message<string, T>
            {
                Key = key,
                Value = value
            };

            if (headers != null)
            {
                message.Headers = [.. headers];
            }

            var deliveryResult = await _producer.ProduceAsync(topic, message);

            Console.WriteLine($"Message delivered to {deliveryResult.TopicPartitionOffset}");
        }
        catch (ProduceException<string, T> e)
        {
            Console.WriteLine($"Delivery failed: {e.Error.Reason}");
        }
    }
}

public class JsonSerializer<T> : ISerializer<T>
{
    public byte[] Serialize(T data, SerializationContext context)
    {
        return System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(data);
    }
}

Acks.All means the leader broker will wait for all in-sync replicas to acknowledge the message. This provides the highest level of durability: the message is only considered "written" when all replicas have it.

Now let's send a message to the movies topic using this class.
Please add the following code in your Program.cs

using KafkaProducer.Configuration;
using KafkaProducer.Models;
using KafkaProducer.Services;
using Microsoft.Extensions.Configuration;

KafkaConfiguration kafkaConfig = GetConfiguration();

var producerService = new ProducerService<Movie>(kafkaConfig);

var topicName = "movies";

Movie movie = CreateMovie();

await producerService.ProduceMessageAsync(topicName, movie.Id.ToString(), movie);

static KafkaConfiguration GetConfiguration()
{
    var builder = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .AddEnvironmentVariables();

    var configuration = builder.Build();

    var kafkaConfig = new KafkaConfiguration();
    configuration.GetSection("KafkaConfiguration").Bind(kafkaConfig);
    return kafkaConfig;
}

static Movie CreateMovie()
{
    return new Movie
    {
        Id = 1,
        Title = "A Separation",
        Director = "Asghar Farhadi",
        ReleaseYear = 2011,
        Genre = "Drama",
        Rating = 8.3
    };
}

Now run the KafkaProducer application.
You should get a success message in the Console.
Also, we can see the message sent in the Kafka UI as below:

Creating a Consumer Application

Our producer successfully published a movie event to Kafka. Now it’s time to build a Consumer application that listens to the movies topic and processes incoming events.

Unlike the producer, which publishes data, a consumer continuously waits for new events and processes them as they arrive.

Create another Console App project in your current solution in Visual Studio and name it KafkaConsumer.

Just like the producer application, install the Confluent.Kafka package.

We will reuse some of the classes/files from the Producer project. Copy Configuration and Models folders with their classes inside (remember to change the namespaces). Also copy the appsettings.json to this project.

ConsumerService

Now create Services folder and create ConsumerService class under it with the following code:

using System.Text.Json;
using Confluent.Kafka;
using KafkaConsumer.Configuration;
using KafkaConsumer.Models;
using Microsoft.Extensions.Options;

namespace KafkaConsumer.Services;

public class ConsumerService
{
    private readonly IConsumer<string, byte[]> _consumer;
    public readonly KafkaConfiguration _kafkaConfig;

    public ConsumerService(IOptions<KafkaConfiguration> kafkaOptions)
    {
        _kafkaConfig = kafkaOptions.Value;
        var topicName = "movies";

        var config = new ConsumerConfig
        {
            BootstrapServers = _kafkaConfig.BootstrapServers,
            GroupId = "movie-consumer-group",
            AutoOffsetReset = AutoOffsetReset.Earliest
        };

        _consumer = new ConsumerBuilder<string, byte[]>(config).Build();

        _consumer.Subscribe(topicName);

        Console.WriteLine("Waiting for movie events...");
        Console.WriteLine();
    }

    public void Consume(CancellationToken cancellationToken)
    {
        try
        {
            var consumeResult = _consumer.Consume(cancellationToken);

            var movie = JsonSerializer.Deserialize<Movie>(consumeResult.Message.Value);

            Console.WriteLine($"Movie received:");
            Console.WriteLine($"  Title: {movie?.Title}");
            Console.WriteLine($"  Director: {movie?.Director}");
            Console.WriteLine($"  Release Year: {movie?.ReleaseYear}");
            Console.WriteLine($"  Genre: {movie?.Genre}");
            Console.WriteLine($"  Rating: {movie?.Rating}");
            Console.WriteLine(new string('-', 50));

        }
        catch (ConsumeException ex)
        {
            Console.WriteLine($"Error occurred while consuming message: {ex.Error.Reason}"); 
        }
    }

    public void Close()
    {
        _consumer.Close();
        Console.WriteLine("Kafka consumer closed.");
    }
}

The consumer starts by creating a ConsumerConfig.

BootstrapServers specifies the address of the Kafka broker that the consumer connects to.

Since we’re running Kafka locally using Docker, the broker is available on port 9092.

Every Kafka consumer belongs to a Consumer Group.

Kafka uses the GroupId to keep track of which messages have already been processed by that group.

If multiple consumer instances share the same Group ID, Kafka distributes partitions among them, allowing the workload to be processed in parallel.

In our example, we’re using a single consumer, but it’s still good practice to assign it to a consumer group.

AutoOffsetReset determines where the consumer starts reading if there is no previously committed offset for the consumer group.

The available options are:

• Earliest – start reading from the beginning of the topic.
• Latest – start reading only newly published messages.

For this article, we use Earliest so that the consumer can read the movie event that we published earlier.

Then the consumer subscribes to the movies topic and consumes the messages.

ConsumerWorker

Now create a Workers folder under the project and add ConsumerWorker with the following code:

using KafkaConsumer.Services;
using Microsoft.Extensions.Hosting;

namespace KafkaConsumer.Workers;

public class ConsumerWorker(ConsumerService consumerService) : BackgroundService
{
    private readonly ConsumerService _consumerService = consumerService;

    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        return Task.Run(() =>
        {
            try
            {
                while (!stoppingToken.IsCancellationRequested)
                {
                    _consumerService.Consume(stoppingToken);
                }
            }
            catch (OperationCanceledException)
            {
                Console.WriteLine("Kafka consumer service is stopping.");
            }
            finally
            {
                _consumerService.Close();
            }
        });
    }
}

As you see, this is a background service that will run continuously and will consume the messages using the ConsumerService we created above.

Program.cs

As the final part of the coding, change the Program.cs with the following code:

using KafkaConsumer.Configuration;
using KafkaConsumer.Services;
using KafkaConsumer.Workers;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

var builder = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .AddEnvironmentVariables();

var configuration = builder.Build();

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices(services =>
    {
        services.Configure<KafkaConfiguration>(configuration.GetSection("KafkaConfiguration"));
        services.AddHostedService<ConsumerWorker>();
        services.AddSingleton<ConsumerService>();
    })
    .Build();

host.Run();

Now if you run the KafkaConsumer project you will see the message sent to the topic before will be consumed as follows:

Congratulations! 😊 You’ve successfully published and read your first Kafka message.

And if you go to Kafka UI you can see the consumer as below:

ConsumerLag = 0 means that the consumer consumed all the messages in the topic.

Publishing and consuming events

Now we will run both KafkaProducer and KafkaConsumer at the same time to see how the events are processed simultaneously.

First, lets add more events to the KafkaProducer so that we can notice they are streamed.

Change the Program.cs as below:

using KafkaProducer.Configuration;
using KafkaProducer.Models;
using KafkaProducer.Services;
using Microsoft.Extensions.Configuration;

KafkaConfiguration kafkaConfig = GetConfiguration();

var producerService = new ProducerService<Movie>(kafkaConfig);

var topicName = "movies";

var movies = CreateMovies();

foreach (var movie in movies)
{
    Console.WriteLine($"Publishing '{movie.Title}'...");

    await producerService.ProduceMessageAsync(
        topicName,
        movie.Id.ToString(),
        movie);

    Console.WriteLine("Published successfully.");
    Console.WriteLine();

    await Task.Delay(1000);
}

Console.WriteLine("All movie events have been published.");

static KafkaConfiguration GetConfiguration()
{
    var builder = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .AddEnvironmentVariables();

    var configuration = builder.Build();

    var kafkaConfig = new KafkaConfiguration();
    configuration.GetSection("KafkaConfiguration").Bind(kafkaConfig);
    return kafkaConfig;
}

static List<Movie> CreateMovies()
{
    return
    [
        new Movie
        {
            Id = 1,
            Title = "Fight Club",
            Director = "David Fincher",
            ReleaseYear = 1999,
            Genre = "Drama",
            Rating = 8.8
        },

        new Movie
        {
            Id = 2,
            Title = "Memento",
            Director = "Christopher Nolan",
            ReleaseYear = 2000,
            Genre = "Mystery",
            Rating = 8.4
        },

        new Movie
        {
            Id = 3,
            Title = "The Sixth Sense",
            Director = "M. Night Shyamalan",
            ReleaseYear = 1999,
            Genre = "Psychological Thriller",
            Rating = 8.2
        },

        new Movie
        {
            Id = 4,
            Title = "Eternal Sunshine of the Spotless Mind",
            Director = "Michel Gondry",
            ReleaseYear = 2004,
            Genre = "Science Fiction / Romance",
            Rating = 8.3
        },

        new Movie
        {
            Id = 5,
            Title = "Children of Heaven",
            Director = "Majid Majidi",
            ReleaseYear = 1997,
            Genre = "Drama",
            Rating = 8.2
        }
    ];
}

Now stop your KafkaConsumer project if it is still running.

To start both the KafkaProducer and KafkaConsumer projects simultaneously, right-click the solution in Visual Studio, select Properties, choose Configure startup projects, and set both projects to Start.

Now, if you click Ctrl+F5, you will see that the messages are sent and received as a stream:

If you go to Kafka UI and check the messages in the topic, you will see that messages are still there:

This proves that the messages are durable in Kafka topics, which was mentioned before.

You can find the source code in this repository.

That’s the end of the post. I hope you found this post helpful and easy to follow. If you have any questions and/or comments, you can share them in the responses section below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

Apache Kafka with .NET: From Core Concepts to a Working Application was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, I will demonstrate how to use Refit library in ASP.NET Core.

We will build an ASP.NET Core Web API that will make external API calls to TMDB API and we will use Refit library while calling this external API.

Our API will have endpoints for the following purposes:

• Search an actor/actress by name
• Get the movies of an actor/actress
• Add a rating to a movie
• Delete a rating from a movie

At the end, the API will look like this:

The sections of this post will be as follows:

• What is Refit?
• Demo Application
  - “Search an actor/actress by name” endpoint (GET)
  - “Get the movies of an actor/actress” endpoint (GET)
  - “Add a rating to a movie” endpoint (POST)
  - “Delete a rating from a movie” endpoint (DELETE)
  - ApiResponse<T> wrapper class

What is Refit?

Refit is an automatic type-safe REST library for .NET. It is heavily inspired by Square’s Retrofit library, Refit turns your REST API into a live interface.

Refit reduces the amount of code required to call APIs by eliminating the need to manually construct HTTP requests and handle responses.

It provides automatic serialization and deserialization of data.

Refit currently supports the following platforms and any .NET Standard 2.0 target:

• UWP
• Xamarin
• Desktop .NET 4.6.1
• .NET 5 / .NET Core
• Blazor
• Uno Platform

The current version of Refit is 7.0.0 and you can see the latest version here.

Refit 6 requires Visual Studio 16.8 or higher, or the .NET SDK 5.0.100 or higher. It can target any .NET Standard 2.0 platform.

Additionally, Refit 6 makes System.Text.Json the default JSON serializer.

You can find the GitHub repository of Refit here.

We will dive into the details of using Refit in our demo project below.

If you are ready, let’s start :)

Demo Application

The API will make external API calls to the TMDb API. If you want to follow along while implementing the application, please go here and get your API Read Access Token to be able to call this API.

Now, let’s start with creating the project.

I will use Visual Studio 2022 and .NET 8 for the demo application.

Open VS 2022 and select Create a new project and then select ASP.NET Core Web API:

Please make sure that Enable OpenAPI support is checked so that we can use Swagger to test our endpoints.

You can delete the default class and the controller (WeatherForecast.cs and WeatherForecastController.cs) as we will not use those.

Add the Refit and Refit.HttpClientFactory Nuget packages to the project:

“Search an actor/actress by name” endpoint (GET)

Now, we will add the implementation for our first endpoint.

First, add Models folder in the project and add Actor and ActorList classes under it:

Next, add Services folder to the project and create ITmdbApi interface with the following code under it:

Now, let’s examine specific features related to Refit here.

Every method must have an HTTP attribute that provides the request method and relative URL. There are six built-in annotations: Get, Post, Put, Delete, Patch and Head. The relative URL of the resource is specified in the annotation.

You can also specify query parameters in the URL. A request URL can be updated dynamically using replacement blocks and parameters on the method. A replacement block is an alphanumeric string surrounded by { and }.

We use the GET attribute for our method as follows:

 [Get("/search/person?query={name}")]
 Task<ActorList> GetActors(string name);

The complete path to query persons in TMDB API is:

https://api.themoviedb.org/3/search/person?query={name}

Since the part https://api.themoviedb.org/3 is common across all endpoints, this part will be defined as base URL later in Program.cs

The relative URL /search/person?query={name} is specified in the annotation. And {name} is matched with the parameter name in the method definition.

If the name of your parameter doesn’t match the name in the URL path, use the AliasAs attribute.

[Get("/search/person?query={name}")]
Task<ActorList> GetActors([AliasAs("name")] string actorName);

You can set one or more static request headers for a request applying a Headers attribute to the method:

[Headers("accept: application/json",
         "Authorization: Bearer")]

As we will use token authentication with TMDB API, we need to add Authorization: Bearer header in the interface. (The rest of the token authentication will be done in Program.cs later)

Note that in Refit unlike in Retrofit, there is no option for a synchronous network request — all requests must be async, either via Task or via IObservable.

So we use Task<ActorList> in the method definition.

Now, we will add the necessary code to Program.cs

Refit has first class support for the ASP.Net Core 2.1 HttpClientFactory. Add a reference to Refit.HttpClientFactory and call the provided extension method in your ConfigureServices method to configure your Refit interface:

builder.Services
    .AddRefitClient<ITmdbApi>(refitSettings)
    .ConfigureHttpClient(c => c.BaseAddress = new Uri("https://api.themoviedb.org/3"));

We configure the Refit client with the base URL https://api.themoviedb.org/3 here.

Notice that we are passing refitSettings parameter to the AddRefitClient

RefitSettings defines various parameters on how Refit should work.

We can use RefitSettings for many purposes to customize our client. In our case, we will use it to pass our API Read Access Token to the client:

var authToken = "***WRITE YOUR API READ ACCESS TOKEN HERE***";
var refitSettings = new RefitSettings()
{
    AuthorizationHeaderValueGetter = (rq, ct) => Task.FromResult(authToken),
};

So, our Program.cs will look like this after those additions:

For TMDB API, access token is static. But if you need to generate a token dynamically, you can do it in a function, say GetTokenAsync , and then you can use it like below:

var refitSettings = new RefitSettings()
{
    AuthorizationHeaderValueGetter = (rq, ct) => GetTokenAsync(),
};

Finally, we will add an API Controller to our project and inject ITmdbApi into it.

Add a new folder called Controllers to the project and then add an empty API Controller named MovieDbController to it and add the following code:

Now, we can test our first endpoint.

Run the project and click Try It Out and then enter the name and/or surname of the actor/actress you want to see and click Execute:

Then we see the results as follows:

Please note the id of the person you want to use as we will use that in the next endpoint.

“Get the movies of an actor/actress” endpoint (GET)

Now, we will implement the endpoint to get the movies of an actor/actress.

First, add the Movie and MovieList classes under the Models folder:

Next, add the following code to ITmdbApi :

 [Get("/person/{actorId}/movie_credits?language=en-US")]
 Task<MovieList> GetMovies(int actorId);

The comparison between parameter name and URL parameter is not case-sensitive, so it will work correctly if you name your parameter as actorid

Finally, add the following code to MovieDbController :

[HttpGet("actors/{actorId}/movies")]
public async Task<MovieList> GetMovies(int actorId)
{
    var response = await _tmdbApi.GetMovies(actorId);
    return response;
}

Now, we can test the endpoint using the id of the actor/actress that we found in the previous endpoint:

Please note the Id of the movie you want to use as we will use that in the next endpoints to add/delete rating.

“Add a rating to a movie” endpoint (POST)

Now, we will add a rating to our favorite movie :)

First, add the Rating and ResponseBody classes under the Models folder:

Next, add the following code to ITmdbApi :

   [Headers("Content-Type: application/json;charset=utf-8")]
   [Post("/movie/{movieId}/rating")]
   Task<ResponseBody> AddRating(int movieId, [Body] Rating rating);

Now, let’s examine what we added here in detail:

We use Post attribute in the data annotation as this is a POST endpoint.

One of the parameters in your method can be used as the body, by using the Body attribute

We use Body attribute here to pass the Rating object.

And we include a new Headers attribute for the Content-Type header. Please notice that we already have a Header attribute for the interface and now we added a new one for the method.

There can be three ways to define a Headers attribute and below is the priority for those when they are defined for the same header:

Headers attribute on the interface (lowest priority)

Headers attribute on the method

Header attribute or HeaderCollection on a method parameter (highest priority)

This redefining behavior only applies to headers with the same name. Headers with different names are not replaced. So, all headers will be included for our code.

Finally, add the following code to the MovieDbController :

[HttpPost("movies/{movieId}/rating")]
public async Task<ResponseBody> AddRating(int movieId, [FromBody] Rating rating)
{
   var response = await _tmdbApi.AddRating(movieId, rating);
   return response;
}

Now, we can add our rating to the movie:

And we get the Success message when we execute it:

Also, we can check that the rating is added to our TMDB account:

“Delete a rating from a movie” endpoint (DELETE)

Now, we will implement the endpoint to delete a rating.

Add the following code to ITmdbApi :

  [Delete("/movie/{movieId}/rating")]
  Task<ResponseBody> DeleteRating(int movieId);

Then, add the following code to IMovieDbController :

 [HttpDelete("movies/{movieId}/rating")]
 public async Task<ResponseBody> DeleteRating(int movieId)
 {
    var response = await _tmdbApi.DeleteRating(movieId);
    return response;
 }

Now, we can use the endpoint after running the project:

If we check our account on the TMDB site then we will see the rating is removed:

ApiResponse<T> wrapper class

There is also a generic wrapper class called ApiResponse<T> that can be used as a return type. Using this class as a return type allows you to retrieve not just the content as an object, but also any metadata associated with the request/response. This includes information such as response headers, the http status code and reason phrase (e.g. 404 Not Found), the response version, the original request message that was sent and in the case of an error, an ApiException object containing details of the error.

Lastly, I want to demonstrate how to use ApiResponse<T> wrapper class as I find it very helpful.

We will change our Search an actor/actress by name GET endpoint.

First, change the GetActors method in the ITmdbApi as follows:

 [Get("/search/person?query={name}")]
 Task<ApiResponse<ActorList>> GetActors(string name);

Then, change the GetActors method in the MovieDbController as below:

 [HttpGet("actors/")]
 public async Task<ActorList> GetActors([FromQuery][Required] string name)
 {
    var result = new ActorList();
    var response = await _tmdbApi.GetActors(name);
    if (response.IsSuccessStatusCode)
    {
        result = response.Content;
    }
    else
    {
        // log $"{response.Error}-{response.Error.Content}";
    }
    return result;
  }

Now, you can run the project and see it still functions properly. In this way, you have more information on the request/response that you can use for controls, exception handling, logging, etc.

You can find the source code on this GitHub repository.

That’s the end of the post. I hope you found this post helpful and easy to follow. If you have any questions and/or comments, you can share them in the responses section below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

Using Refit in .NET was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.

.NET 6 is the latest LTS (Long Term Support) release currently and will be supported until November 12, 2024.

This API will manage movie records stored in a relational database (SQL Server) as described in the table below:

The sections of this post will be as follows:

• What is REST?
• Creating a Web API Project
• Adding a Model
• Adding a Database Context
• Creating Database with Migrations
• Creating API Controller and Methods

We need the following tools installed on our computer:

• Visual Studio 2022
• .NET 6.0 SDK
• Microsoft SQL Server Express

If you are ready, let’s get started.

What is REST?

RESTful APIs conform to the REST architectural style.

REST, or REpresentational State Transfer, is an architectural style for providing standards between computer systems on the web, making it easier for systems to communicate with each other.

REST relies on client-server relationship. This essentially means that client application and server application must be able to evolve separately without any dependency on each other.

REST is stateless. That means the communication between the client and the server always contains all the information needed to perform the request. There is no session state in the server, it is kept entirely on the client’s side.

REST provides a uniform interface between components. Resources expose directory structure-like URIs.

REST is not strictly related to HTTP, but it is most commonly associated with it. There are four basic HTTP verbs we use in requests to interact with resources in a REST system:

• GET — retrieve a specific resource (by id) or a collection of resources
• POST — create a new resource
• PUT — update a specific resource (by id)
• DELETE — remove a specific resource by id

In a REST system, representations transfer JSON or XML to represent data objects and attributes.

REST has had such a large impact on the Web that it has mostly displaced SOAP-based interface design because it’s a considerably simpler style to use.

Now that we have made a quick review of REST, we can continue with the implementation.

Creating a Web API Project

Open Visual Studio 2022 and select Create a new project and then select ASP.NET Core Web API:

and give a name to your project in the following screen and then click Next.

In the next screen, select .NET 6.0 as the framework and click Create:

At this point we have a starter project as follows:

In the Program.cs we see that Swagger support is added automatically to our project:

And also Swashbuckle.AspNetCore NuGet package is added as a dependency.

For more information on Swagger, see ASP.NET Core web API documentation with Swagger / OpenAPI.

Now, let’s run (Ctlr+F5) the project to see the default output. When the browser opens and the Swagger UI is shown, select the GET method in the WeatherForecast part and then select Try It Out and Execute:

Also, you can use the curl URL shown in the Swagger UI for this method and see the result of the URL in the browser:

When we run the application, the default URL comes from the launchSettings.json:

And the result values come from the GET method of the WeatherForecastController:

As you see, values here are hard coded and randomness is added to generate different values.

In our Web API, we will create our own records in an SQL server database and will be able to view, update and delete them through REST API endpoints.

Adding a Model

Now, we will implement our data model class.

In Solution Explorer, right-click the project. Select Add -> New Folder and name the folder Models.

Then right-click the Models folder and select Add->Class. Name the class Movie.cs and click Add.

Next, add the following properties to the class:

The Id field is required by the database for the primary key.

Entity Framework Core

We will use our model with Entity Framework Core (EF Core) to work with a database.

EF Core is an object-relational mapping (ORM) framework that simplifies the data access code. Model classes don’t have any dependency on EF Core. They just define the properties of the data that will be stored in the database.

In this post, we will write the model classes first and EF Core will create the database. This is called Code First Approach.

Let’s add the EF Core NuGet packages to the project. Right-click on the project and select Manage NuGet Packages… and then install the following packages:

Adding a Database Context

The database context is the main class that coordinates Entity Framework functionality for a data model. This class is created by deriving from the Microsoft.EntityFrameworkCore.DbContext class.

Now, right-click the Models folder and select Add ->Class. Name the class MovieContext and click Add. Then add the following code to the class:

UPDATE (26.08.2023): You need to addpublic keyword to the above line as below:

public DbSet<Movie> Movies { get; set; } = null!;

The preceding code creates a DbSet<Movie> property for the entity set.

In Entity Framework terminology, an entity set typically corresponds to a database table and an entity corresponds to a row in the table.

The name of the connection string is passed into the context by calling a method on a DbContextOptions object. For local development, the ASP.NET Core configuration system reads the connection string from the appsettings.json file.

We need to add our connection string to the appsettings.json. I will use the local SQL server instance in my machine and we can define the connection string as follows:

You can change the database name if you want.

Dependency Injection

ASP.NET Core is built with Dependency Injection (DI). Services (such as the EF Core DB context) are registered with DI during application startup. Components that require these services are provided with these services via constructor parameters.

Now, we will register our database context to the built-in IOC container. Add the following code to Program.cs:

Creating Database with Migrations

Now, we will create the database using the EF Core Migrations feature.

Migrations lets us create a database that matches our data model and update the database schema when our data model changes.

First, we will add an initial Migration.

Open Tools -> NuGet Package Manager > Package Manager Console(PMC) and run the following command in the PMC:

Add-Migration Initial

The Add-Migration command generates code to create the initial database schema which is based on the model specified in the MovieContext class. The Initial argument is the migration name and any name can be used.

After running the command, a migration file is created under the Migrations folder:

As the next step, run the following command in the PMC:

Update-Database

The Update-Database command runs the Up method in the Migrations/{time-stamp}_Initial.cs file, which creates the database.

Now, we will check the database created. Open View -> SQL Server Object Explorer.

You will see the newly created database as below:

As you see, the Movie table and the Migrations History table are created automatically. Then a record is inserted into the migration history table to show the executed migrations on the database.

Creating API Controller and Methods

In this section, we will create the Movies API Controller and add the methods to it, and also will test those methods.

Let’s add the controller first. Right-click on the Controller folder and select Add -> Controller.. and then select API Controller - Empty as below:

Click Add and give a name to your controller on the next screen.

MoviesController is created as below:

As you see, the class is decorated with the [ApiController] attribute. This attribute indicates that the controller responds to web API requests.

MoviesController class inherits from ControllerBase.

Next, we will inject the database context mentioned in the previous section through the constructor of the controller. Add the following code:

Now, we will add CRUD (create, read, update, and delete) action methods to the controller. Let’s start with the GET methods.

GET Method

Add the following code to the MoviesController:

GetMovies method returns all the movies and GetMovie(int id) method returns the movie having the Id given as input. They are decorated with the [HttpGet] attribute which denotes that a method responds to an HTTP GET request.

These methods implement two GET endpoints:

• GET /api/Movies
• GET /api/Movies/{id}

We can test the app by calling the two endpoints from a browser as follows:

• https://localhost:{port}/api/movies
• https://localhost:{port}/api/movies/{id}

The return type of the GetMovie methods is ActionResult<T> type. ASP.NET Core automatically serializes the object to JSON and writes the JSON into the body of the response message. The response code for this return type is 200, assuming there are no unhandled exceptions. Unhandled exceptions are translated into 5xx errors.

Routing and URL Paths

The URL path for each method is constructed as follows:

Start with the template string in the controller’s Route attribute (Route("api/[controller]")). Then replace [controller] with the name of the controller, which by convention is the controller class name minus the Controller suffix. For this sample, the controller class name is MoviesController, so the controller name is movies.

ASP.NET Core routing is case insensitive.

Testing the GetMovie Method

Now we will test these endpoints. Before that, let’s insert some movie records into our table.

Go to the SQL Server Object Explorer and right-click the Movies table and select View Data:

Then add some movie records manually to the table:

We do not need to add data for the Id column as SQL Server automatically handles this for us.

Now, we can test the GET endpoints. Start (Ctlr+F5) the application:

Select the first GET method and click Try it out -> Execute:

This shows all of the movies in the application.

Next, click the second GET method and click Try it out and enter one of the Ids above in the id field and click Execute:

If no item matches the requested Id, the method returns a 404 NotFound error code.

POST Method

Add the following code to the MoviesController:

PostMovie method creates a movie record in the database. The preceding code is an HTTP POST method, as indicated by the [HttpPost] attribute. The method gets the value of the movie record from the body of the HTTP request.

The CreatedAtAction method:

• Returns an HTTP 201 status code, if successful. HTTP 201 is the standard response for an HTTP POST method that creates a new resource on the server.
• Adds a Location header to the response. The Location header specifies the URI of the newly created movie record.
• References the GetMovie action to create the Location header's URI.

Testing the PostMovie Method

Start the application and then select the POST method in the Movies section.

Click Try it out and enter the movie information that you want to add in the request body:

and click Execute.

Response status code is 201 (Created) and a location header is added to the response as seen below:

We can paste this location URL in the browser and see the response there too:

Also, we can check this record from the Movies table in our local database:

PUT Method

Add the following code to the MoviesController:

PutMovie method updates the movie record with the given Id in the database. The preceding code is an HTTP PUT method, as indicated by the [HttpPut] attribute. The method gets the value of the movie record from the body of the HTTP request. You need to supply the Id both in the request URL and the body and they have to match. According to the HTTP specification, a PUT request requires the client to send the entire updated entity, not just the changes.

The response is 204 (No Content) if the operation is successful.

Testing the PutMovie Method

Start the application and then select the PUT method in the Movies section.

Click Try it out and enter the movie information that you want to update in the request body and the Id of the movie in the id field:

and then click Execute.

We can check the updated state of the movie from GET method with Id in the Swagger UI or directly from the browser as below:

We can see the updated info in the database as well:

If we try to update a record that does not exist in the database we get 404 Not Found error:

DELETE Method

Add the following code to the MoviesController:

DeleteMovie method deletes the movie record with the given Id in the database. The preceding code is an HTTP DELETE method, as indicated by the [HttpDelete] attribute. This method expects Id in the URL to identify the movie record we want to delete.

Testing the DeleteMovie Method

Start the application and then select the DELETE method in the Movies section.

Click Try it out and enter the Id of the movie you want to delete in the id field:

and then click Execute.

We do not need to supply a request body as you might have noticed. The response status is 204 No Content.

If we try to get this movie record using the browser we get 404 Not Found error as expected:

We can check as well from the database that the record is deleted:

This is where this post ends. You can find the full project in this GitHub repository.

I hope you found this post helpful and easy to follow. Please let me know if you have any corrections and/or questions in the comments below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

References

https://www.codecademy.com/articles/what-is-rest

https://developer.ibm.com/articles/ws-restful/

https://ninenines.eu/docs/en/cowboy/2.6/guide/rest_principles/

https://restfulapi.net/rest-architectural-constraints/

https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-6.0

Build a RESTful Web API with ASP.NET Core 6 was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.

I will use the following tools & technologies:

• ASP.NET Core 3.1
• Visual Studio 2019
• Redis

Let’s start with the definition of caching.

Caching

A cache is a hardware or software component that stores data so that future requests for that data can be served faster; the data stored in a cache might be the result of an earlier computation or a copy of data stored elsewhere. A cache hit occurs when the requested data can be found in a cache, while a cache miss occurs when it cannot. Cache hits are served by reading data from the cache, which is faster than recomputing a result or reading from a slower data store; thus, the more requests that can be served from the cache, the faster the system performs.[1]

Caching can significantly improve the performance and scalability of an app by reducing the work required to generate content. Caching works best with data that changes infrequently and is expensive to generate. Caching makes a copy of data that can be returned much faster than from the source. Apps should be written and tested to never depend on cached data.[2]

Now, I will present the API in which we will use caching.

About the API

The API returns the movies of an actor/actress given as a parameter and looks like below:

It makes an external API call to the TMDb API and the controller code is shown below:

As you see, an external API call is made for every query. However, there will be cases like the same actor/actress is queried several times by the same or different users. So, we can increase the performance of this application by using caching mechanism instead of calling the external API again and again for the same actors/actresses.

First, let’s see how we can implement in-memory caching in this API.

In-memory Caching

ASP.NET Core supports several different caches. The simplest cache is based on the IMemoryCache. IMemoryCache represents a cache stored in the memory of the web server. Apps running on a server farm (multiple servers) should ensure sessions are sticky when using the in-memory cache. Sticky sessions ensure that subsequent requests from a client all go to the same server.[3]

In-memory caching is a service that’s referenced from an app using Dependency Injection. So, we first need to register this service to the built-in IoC container of ASP.NET Core by modifying ConfigureServices method of Startup.cs as below:

Then we inject IMemoryCache to the constructor:

And we change GetMovieList method as below:

As you see in the above code, we first check if the movie list exists in the cache and return from there if it does. Otherwise, we call the external API and cache the result.

Besides, we set the cache expiration options which are explained below:

SlidingExpiration: Gets or sets how long a cache entry can be inactive (e.g. not accessed) before it will be removed. This will not extend the entry lifetime beyond the absolute expiration (if set).

AbsoluteExpiration: Gets or sets an absolute expiration date for the cache entry.

A cached item set with a sliding expiration only is at risk of becoming stale. If it’s accessed more frequently than the sliding expiration interval, the item will never expire. Combine a sliding expiration with an absolute expiration to guarantee that the item expires once its absolute expiration time passes.[4]

In the next section, we will implement distributed caching for the same controller.

Distributed Caching

A distributed cache is a cache shared by multiple app servers, typically maintained as an external service to the app servers that access it. A distributed cache can improve the performance and scalability of an ASP.NET Core app, especially when the app is hosted by a cloud service or a server farm.

A distributed cache has several advantages over other caching scenarios where cached data is stored on individual app servers. When cached data is distributed, the data:

• Is coherent (consistent) across requests to multiple servers.
• Survives server restarts and app deployments.
• Doesn’t use local memory.[5]

There are two main disadvantages of the shared caching approach:

• The cache is slower to access because it is no longer held locally to each application instance.
• The requirement to implement a separate cache service might add complexity to the solution.[6]

We will use Redis-based distributed caching in our API. So let’s continue with what Redis is and how to install it.

Redis

The name Redis means REmote DIctionary Server.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams.[7]

Now, I will show how to install Redis in a Windows machine using Chocolatey. (If you don’t have Chocolatey you can install it from here.)

First, install the Chocolatey Redis package.

Then run redis-server from a command prompt.

Now, the Redis server is up and running. We can test it using the redis-cli command.

Open a new command prompt and run redis-cli on it and try the following commands:

Now that we installed the Redis-server and saw that it is working properly, we can modify the API to use Redis-based distributed caching.

Modify the API

ASP.NET Core provides IDistributedCache interface to interact with the distributed caches including Redis.

• Get , GetAsync : Accepts a string key and retrieves a cached item as a byte[] array if found in the cache.
• Set , SetAsync : Adds an item (as byte[] array) to the cache using a string key.
• Refresh , RefreshAsync : Refreshes an item in the cache based on its key, resetting its sliding expiration timeout (if any).
• Remove , RemoveAsync : Removes a cache item based on its string key.

To use a Redis distributed cache, we need to add a package reference to Microsoft.Extensions.Caching.StackExchangeRedis package in our project.

Then we add the following line to ConfigureServices method of the Startup.cs for configuration the cache implementation using a RedisCache instance:

Our approach in modifying the controller will be very similar to what we did in the in-memory caching section.

First, we inject IDistributedCache to the controller:

Then we change the GetMovieList method as follows:

As mentioned above, GetAsync and SetASync methods work with byte[] arrays, so I encoded the serialized movies list to byte[] array. Also, I want to mention that there are extension methods where you do not need to convert the value to byte[] array and these are GetStringAsync and SetStringAsync.

You can find the source code in this GitHub repository.

That’s the end of the post. I hope you found this post helpful and easy to follow. If you have any questions and/or comments, you can share them in the responses section below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

In-memory & Distributed (Redis) Caching in ASP.NET Core was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.

I will use the following tools:

• Google Cloud Console
• Visual Studio 2017 Community Edition
• Google Cloud Tools for Visual Studio
• pgAdmin

The sections of this post will be as follows:

• About the App
• Sign up for Google Cloud
• Create the GCP Project
• Cloud SQL
• Modify the App and Initialize the Database
• Check the Database (pgAdmin)
• Test
• Publish To Google Cloud
• Clean up the GCP Project

In the previous post, I demonstrated how to deploy an ASP.NET Core web application starter project to GCP. As it has no database access, it is a simpler version of this one and I recommend you start from there if this is your first deployment to GCP.

If you are ready, let’s get started.

About the App

The application that we will deploy manages a database of TV Shows and its main page looks like below:

The app is an ASP.NET Core MVC web application and it uses EF Core to perform CRUD operations on a PostgreSQL database.

You can download the source code from this GitHub repository.

Sign up for Google Cloud

If you don’t have a Google Cloud account, go to Google Cloud web site and click Get started for free button. Sign up with your Gmail account and get $300 (for free) to spend on Google Cloud Platform over the next 12 months.

Creating the GCP Project

Next, go to Google Cloud Console and type App Engine in the search box and click Create in the dashboard of the App Engine to create a new project:

Name your project and then select your region and enable billing in the next page:

Cloud SQL

Now, we will create our PostgreSQL instance in the Cloud SQL.

Cloud SQL is a fully-managed database service that makes it easy to set up, maintain, manage, and administer your relational databases on Google Cloud Platform. You can use Cloud SQL with either MySQL or PostgreSQL.

First, select APIs & Services -> Library

and in the next page search for Cloud SQL API and Enable this API:

Next, search for Cloud SQL and click Go To Cloud SQL in the following page:

Click Create Instance in the next page:

and then choose PostgreSQL.

On the next page, give an ID to your instance and set your password for the default (postgres) user.

We will use this Id and password to connect to this instance later in the tutorial.

Next, the Compute Engine API creates the instance and it takes a few minutes.

After you see the green tick on the instance ID, click on the instance to go to Instance Details page:

In the Overview tab, you can see the IP and connection name for the instance:

Next, go to the Connections tab and click Add Network:

• Enter all for the name.
• Enter 0.0.0.0/0 for the network.
• Click Done, then Save.

Security Issues

As you see, we gave access to public IPs to make login attempts to the instance. SSL encryption is recommended when using Public IP to connect to the instance.

For the sake of simplicity, I will not implement this part in this tutorial. If you want to implement this, please follow the instructions here. Besides, you will need to modify your app too. If you need help in this part, let me know in the comments.

Modify the App and Initialize the Database

In Visual Studio, open the application and modify the connection string in appsettings.json:

"ConnectionStrings": {

"TvShowsContext": "Uid=[YOUR_USER_NAME];Pwd=[YOUR_PASSWORD];Host=[YOUR_INSTANCE_IP];Database=TvShowsContext-fe1316f5-48e2-4739-ae44-eccbe217ab10"

}

If you haven’t created your migration file already, run the following command in the Package Manager Console:

Add-Migration Initial

and then run the below command to create the database on the instance:

Update-Database

Check the Database (pgAdmin)

Now, we will check the database created on Cloud SQL using pgAdmin.

Launch pgAdmin and then select Servers -> Create -> Server:

and then fill in the fields shown in the red box below:

As you see below, our database and tables were created on the Cloud SQL:

Test

Now, we will test the application on our local computer.

In this test configuration, our application server is on localhost and the database server is on the cloud.

Run the application on the Visual Studio and click TVShowsApp on the web page and then Create New on the next page.

After creating our first record, it is shown below on the main page:

We can check the record created using pgAdmin as well:

Publish to Google Cloud

Now, we will publish our application to Google Cloud.

First, install Google Cloud Tools for Visual Studio if you don’t have it already.

Open Visual Studio and go to Tools -> Extensions and Updates. Search for Google Cloud Tools for Visual Studio and click Install (It’s already installed on my computer):

As a side note, Cloud Tools for Visual Studio is not supported in VS 2019. So, I am using VS 2017 for this tutorial.

Next, we will link our application to our GCP project.

Open Tools -> Google Cloud Tools -> Manage Accounts:

and select Add account. Login to the account that you used to register to the GCP.

You will see your account logged at the top right corner of Visual Studio. There is a No Project label near this account and click that and click Select Project:

Select the project that you created above.

Enable the following APIs from Google Cloud Console:

Next, right-click on the project in Visual Studio and select Publish To Google Cloud. Then select App Engine Flex in the next dialog.

After the deployment operation is completed, the web page is launched automatically and the application’s URL is as follows:

https://[Your-project_id].appspot.com

I created a new record as we did in the previous section and the main page of the application running on the cloud looks like below now:

Again, we can check the new record from the database using pgAdmin:

Clean up the GCP Project

To avoid incurring charges, we can delete our GCP project to stop billing for all the resources used within the project.

In the Google Cloud Console, search for Manage Resources. In this menu, select your project and click Delete:

That’s the end of the post. I hope you found this post helpful and easy to follow. If you have any questions and/or comments, please let me know in the responses section below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

References

https://cloud.google.com/appengine/docs/flexible/dotnet/using-cloud-sql-postgres

Deploy an ASP.NET Core App with EF Core and PostgreSQL to Google Cloud was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, I will show how to use Google Cloud Storage (GCS) to store and serve images of an ASP.NET Core web application.

The sections of this post will be as follows:

• About the App
• Sign up for Google Cloud
• Google Cloud Storage
• Modify the App
• Test the App
• Clean up the Resources

About the App

The application that we will modify manages a database of TV Shows and its main page looks like below:

The app is an ASP.NET Core MVC web application and it uses EF Core to perform CRUD operations on a SQL Server database. You can download the source code from this Github repository.

If you are interested in the development process of this application, you can check this post.

In the current version, the image of the TV show needs to exist in the wwwroot/images folder of the project and is entered as shown below:

In the modified version, the user will select the image file to be uploaded and the file will be uploaded to Google Cloud Storage in the create action.

Let’s start by performing the necessary actions in the Google Cloud Platform (GCP).

Sign up for Google Cloud

If you don’t have a Google Cloud account, go to Google Cloud web site and click Get started for free button. Sign up with your Gmail account and get $300 (for free) to spend on Google Cloud Platform over the next 12 months.

Google Cloud Storage

In this section, we will do the necessary operations to use Google Cloud Storage service of the GCP for our application.

Cloud Storage provides worldwide, highly durable object storage that scales to exabytes of data. You can access data instantly from any storage class, integrate storage into your applications with a single unified API, and easily optimize price and performance.

Cloud Storage is typically used to store unstructured data.

First, we will create a new GCP project in the Google Cloud Console. (You can use an existing project too). Select the dropdown box shown in the red box and then click New Project.

Give your project a name and then click Create.

Next, we will create a bucket to store our images.

Buckets are the basic containers that hold your data in Cloud Storage.

Write Storage in the search box and then click Storage.

Click Create Bucket on the next screen. Then,

• give a unique name for your bucket,
• choose Standard for Storage class,
• choose a suitable Location for you,
• click Create.

After the creation is completed, we will see our bucket as follows:

Next, we will make the bucket publicly readable so it can serve files:

• Select the Permissions tab shown in the red box above.
• Click the Add members button.
• In the New members field, enter allUsers.
• In the Roles drop-down, select the Storage sub-menu, and click the Storage Object Viewer option as shown below:

• Click Save.

Now, we will set up a service account and download credentials to run our application locally.

Enter APIs & Services in the search box of the cloud console. Then,

• click Credentials on the left pane,
• click Create credentials,
• select Service account key.

In the next dialog, create the new service account as shown below:

After this operation, a new JSON file is created and we download this file to our computer to use in our project in the next section.

Modify the App

In this section, we will perform the modifications listed below in our application so that we will be able to upload the image of the TV show that will be used in the application:

• Modify the model
• Modify the views
• Edit the application settings
• Install the Google Cloud Storage package
• Create a new class for GCS operations
• Edit Startup.cs
• Modify the controller

Modify the model

Change the TvShow model as seen below:

We added two new properties to the model:

• ImageFile will be used to get the image file to be uploaded
• ImageStorageName will be used to keep the name of the object stored in the GCS bucket related to the image file uploaded.
• MaxFileSize and PermittedExtensions are custom validation attributes used to limit the maximum file size and extensions of the image file. (You can find the related classes in the CustomValidationAttributes folder of the final version of the project which will be shared at the end of the tutorial)

Next, we will run the following commands in the Package Manager Console to update the database.

Add-Migration ImageFile
Update-Database

Modify the views

In the Create.cshtml, add the parts shown in red boxes below and delete the part related to ImageUrl :

Modify the Edit.cshtml as shown below:

Next, we will add style attribute to the ImageUrl to make the height and width of image standard in the other views.

In the Delete.cshtml and Details.cshtml, change the part related to ImageUrl with the following:

<dd class="col-sm-10">

<img src="@Url.Content(Model.ImageUrl)" alt="Image" style="width: 150px; height: 220px;"/>

</dd>

In the Index.cshtml, change the part related to ImageUrl with the following:

@if (item.ImageUrl != null)

{

<img src="@Url.Content(item.ImageUrl)" alt="Image" style="width: 150px; height: 220px;"/>

}

Edit the application settings

Add the following key-value pairs to appsettings.json and change the values with the values of the credential file path and GCS bucket that you created in the previous section:

Install the Google Cloud Storage package

Right-click on the project and select Manage Nuget Packages… and then install the following package:

Create a new class for GCS operations

Now, we will create a new class that will manage the file upload and delete operations in our GCS bucket using the client library that we installed above.

Right-click on the project and create a new folder called CloudStorage. Then add the following ICloudStorage interface to this folder.

Next, add a class called GoogleCloudStorage which implements the ICloudStorage under this folder:

This class gets the credential JSON file name and the GCS bucket name from the application settings that we defined previously.

As seen in the implementation, it has two methods:

• UploadFileAsync: Uploads the file given with the parameter imageFile to the GCS with the given fileNameForStorage as object name.
• DeleteFileAsync: Deletes the file with the given fileNameForStorage as object name from the GCS bucket.

We will use these methods in the Create, Edit and Delete action methods of the TvShowsController.

Edit Startup.cs

Before using the GoogleCloudStorage class in the TvShowsController, we need to register it as a service in Startup.cs to the built-in IoC container as it will be injected to the controller via constructor injection.

Add the following line to the end of the ConfigureServices method:

services.AddSingleton<ICloudStorage, GoogleCloudStorage>();

Modify the controller

Now, we will modify Create, Edit and Delete action methods of the TvShowsController.

First, change the following part for the dependency injection:

Second, add the following new methods to the controller class:

FormFileName method creates a new object name in the format below:

{tv-show-title}-{current-datetime}.{file-extension}

Then, this new name is passed as a parameter to the UploadFile method and it is used as the object name in the GCS bucket. This object name will also be stored in the TvShow.ImageStorageName field and will be used to delete the object from the GCS bucket if needed.

UploadFile method calls the UploadFileAsync method that we defined in the GoogleCloudStorage class previously. Notice that after the file is uploaded to GCS, the public URL to this file is returned and we will use this URL to serve the file directly from GCS. So we store this value in the TvShow.ImageUrl field to show the poster in the views of the application.

Next, change the Create method as shown below:

As seen above, when we create a new TV Show, we also upload the image file (if the user chooses one) to the GCS using the UploadFile method.

Next, change Edit method as shown in the below image:

In the Edit method, we first check if there is a related poster for this TV Show and if it exists we delete that from the GCS. Then, we upload the new image to the GCS.

Lastly, add the following code to the DeleteConfirmed method:

You can find the last version of the project in this GitHub repository.

Test the App

Now, we will test the application and see if everything works as expected.

First, let’s create a new TV Show:

As you see above, we can now choose the image file from our computer. After clicking the Create button, the new record including the fields related to the image is created in the database as shown below:

We can also check that the object is created in our bucket in the GCS:

Notice that the ImageStorageName in the table and the Name field in the GCS bucket match.

In the Index view of the application, the poster of the image is served from the GCS using the ImageUrl field in the table.

Next, we will test the Delete view:

After clicking the Delete button, if we refresh the GCS bucket we see that the image object is deleted here as well.

You can check the Edit view and see that everything works as expected.

Clean up the Resources

To avoid incurring charges, we can delete our GCP project to stop billing for all the resources used within the project.

In the Google Cloud Console, search for Manage Resources. In this menu, select your project and click Delete.

That’s the end of the post. I hope you found this post helpful and easy to follow. If you have any questions and/or comments, please let me know in the responses section below.

And if you liked this post, please clap your hands 👏👏👏

Bye!

References

https://cloud.google.com/storage/

https://cloud.google.com/storage/docs/quickstart-console

https://cloud.google.com/appengine/docs/flexible/dotnet/using-cloud-storage

Using Google Cloud Storage in ASP.Net Core was originally published in .NET Core on Medium, where people are continuing the conversation by highlighting and responding to this story.