ASP.NET Core 9 Web API Cookbook

Practical Data Access in ASP.NET Core Web APIs

ASP.NET Core 9 Web API Cookbook aims to be your comprehensive toolbox for building web APIs with ASP.NET Core 9—Microsoft’s latest version of ASP.NET Core. Our goal is to give you practical recipes you can use on the job. This cookbook provides step-by-step solutions to workaday problems such as implementing efficient data access patterns and secure authentication flows, to leveraging cutting-edge technologies such as .NET Aspire for cloud orchestration and distributed caching with HybridCache.

Throughout multiple chapters, we will cover the entire spectrum of RESTful APIs—from implementing robust testing strategies in Chapter 7 to working with .NET Aspire in Chapter 9 and tackling caching in Chapter 10. While RESTful APIs using JSON are the primary focus, the ASP.NET Core 9 web API offers additional powerful technologies for building APIs, such as GraphQL for efficient querying, gRPC for high-performance communication, and SignalR for real-time functionality. If you are transitioning from a legacy .NET Framework background, don’t miss the chapter on middleware. Middleware is perhaps the most significant architectural change from the module and handler system in .NET Framework.

In this chapter, we will explore best practices for retrieving data from an ASP.NET Core web API, including techniques for communicating from the data access layer to the consumer of the API. We will focus on paging and retrieving data with Entity Framework (EF) Core. We will also cover two of the three new LINQ methods, CountBy and AggregateBy, to return additional grouping information on your data.

The following recipes will be covered in this chapter:

Creating a mock database for EF Core with Bogus
Using ProblemDetails to return more robust error information
Creating a categories endpoint using the new LINQ CountBy
Implementing KeySet pagination
Configuring a CORS policy to expose pagination metadata
Implementing efficient first- and last-page access with EF Core
Testing the API in PowerShell
Using the new AggregateBy LINQ method to return the average price per category

Creating a mock database for EF Core with Bogus

Let’s start by creating a basic web API that will serve as the foundation for the rest of the projects in this chapter. We will use SQLite’s in-memory database provider with EF Core, eliminating the need for database files or server connections. The API’s database will be populated with mock data generated by Bogus.

Getting ready

To begin, you will need the following:

The .NET 9 SDK, which can be downloaded from https://dotnet.microsoft.com/en-us/download/dotnet/9.0

How to do it…

Open the terminal and create a new web API:

dotnet new webapi -o mockAPI -f net9.0 --no-https --auth none

Navigate to the project directory and create a new .gitignore file:
```
dotnet new gitignore
```

Install EF Core and its SQLite provider:

dotnet add package Microsoft.EntityFrameworkCore
dotnet add package Microsoft.EntityFrameworkCore.Sqlite

Install Bogus for creating mock data. New in .NET 9, we also have to manually add Swagger support:
```
dotnet add package Bogus
dotnet add package Swashbuckle.AspNetCore.SwaggerUI
```

Create a folder named Models. Create a file called Product.cs and fill in the following Product class:

namespace mockAPI.Models;
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; } = string.Empty;
    public decimal Price { get; set; }
    public int CategoryId { get; set; }
}

Create a sibling folder called Data. In that folder, create your AppDbContext.cs file. Fill in a AppDbContext class, which will inherit from DbContext:

using Microsoft.EntityFrameworkCore;
using mockAPI.Models;
namespace mockAPI.Data;
public class AppDbContext : DbContext
{
    public AppDbContext(DbContextOptions<AppDbContext> options):
        base(options) { }

Still inside the AppDbContext class, on the next line, define the 'DbSet' property to use your new Products class:
```
public DbSet<Product> Products { get; set; }
```

On the next line, define the OnModelCreating method:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
     base.OnModelCreating(modelBuilder);
     modelBuilder.Entity<Product>(entity =>
     {
         entity.HasKey(e => e.Id);
         entity.Property(e => e.Name).IsRequired();
         entity.Property(e => e.Price).HasColumnType(
             "decimal(18,2)");
     });
}

Open the Program.cs file. Delete all the boilerplate code that .NET generated; we are going to start from scratch.

At the top of Program.cs, import our new Data namespace, as well as the namespace for models and Bogus itself:

using mockAPI.Data;
using mockAPI.Models;
using Microsoft.EntityFrameworkCore;
using Microsoft.Data.Sqlite;
using Bogus;

Next, we will create the builder, register the OpenAPI service, and create a connection to our in-memory database that will persist for the application’s lifetime:
```
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddOpenApi();
var connection = new SqliteConnection("DataSource=:memory:");
connection.Open();
```

After creating the SQLite connection, we need to add the DbContext registration:

builder.Services.AddDbContext<AppDbContext>(options =>
    options.UseSqlite(connection));

On the next line, inject a scoped service that opens a connection to the in-memory database and then confirms that the connection has been created:

var app = builder.Build();
using (var scope = app.Services.CreateScope())
{
    var services = scope.ServiceProvider;
    var context = services.GetRequiredService<AppDbContext>();
    context.Database.EnsureCreated();
}

Service lifetime selection

We need to register a scoped service to interact with EF Core’s DbContext. This is counterintuitive; it might seem like a singleton service would be a more appropriate lifetime for working with a database. However, EF Core's DbContext is designed to be short-lived and is not thread-safe for concurrent operations.

Expand the scoped service to seed your in-memory database with fake data, only if your database context is empty:

    if (!context.Products.Any())
    {
        var productFaker = new Faker<Product>()
            .RuleFor(p => p.Name, f => f.Commerce.ProductName())
            .RuleFor(p => p.Price, f => f.Finance.Amount(
                 50,2000))
            .RuleFor(p => p.CategoryId, f => f.Random.Int(
                 1,5));
        var products = productFaker.Generate(10000);
        context.Products.AddRange(products);
        context.SaveChanges();
    }
}

Let’s add a minimal API endpoint we can use for testing:

app.MapGet("/products", async (AppDbContext db) => 
    await db.Products.OrderBy(p =>
        p.Id).Take(10).ToListAsync());

Before we run the application, let’s manually add SwaggerUI support:

app.MapOpenApi();
if (app.Environment.IsDevelopment())
{
    app.UseSwaggerUI(options =>
    {
        options.SwaggerEndpoint("/openapi/v1.json", "v1");
    });
}
app.Run();

Run the application. You can visit http://localhost<yourport>/swagger/index.html and click on the Products endpoint to see our fake data generated by Bogus:
```
dotnet run
```

How it works…

You registered your AppDbContext with the service provider at startup, which is the standard way to integrate EF Core into ASP.NET Core dependency injection. This allows the database context to be available for your controllers, services, and so on.

You also added a scoped service provider that checks whether your database is empty. The scoped lifetime ensures that a new AppDbContext is created for each request, preventing any data inconsistencies that can plague singleton instances of database connections. If the database is empty, it will be seeded using the Faker<T> class from Bogus.

We also used the SQLite in-memory database provider for EF Core. This allows us to create a database entirely in memory without requiring an external SQLite file. While EF Core also includes an InMemory database provider, it is considered a legacy option and is not recommended for testing. Unlike the InMemory provider, SQLite’s in-memory database supports transactions and raw SQL, making it a closer approximation of a real-world database.

Using ProblemDetails to return more robust error information

In this recipe, we will enhance our API’s error handling by leveraging ProblemDetails, a standardized way to provide detailed error information as defined in the HTTP specification.

ProblemDetails allows ApiController to transform error status codes into structured and informative error responses. ProblemDetails is part of the HTTP specification and a great way to return additional error information if something goes wrong with an endpoint. We will explore how to create custom ProblemDetails objects and customize them to include meaningful details, such as including a traceId from HttpContext within the ProblemDetails object itself.

Getting ready

This recipe uses a starter project that includes a basic controller with endpoints already set up and configured. This recipe is not a direct continuation of the preceding recipe.

You can clone the starter project from here: https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/problemDetails.

How to do it…

Open the Program.cs file. On the line right after AddControllers(), let’s register customization options using AddProblemDetails():

builder.Services.AddProblemDetails(options =>
    options.CustomizeProblemDetails = (context) =>
    {
        var httpContext = context.HttpContext;
        context.ProblemDetails.Extensions["traceId"] = Activity.
            Current?.Id ?? httpContext.TraceIdentifier;
        context.ProblemDetails.Extensions["supportContact"] = 
            "support@example.com";

Starting on the next line, let’s enhance our ProblemDetails by adding custom messages for different status codes:

        if (context.ProblemDetails.Status == StatusCodes.
            Status401Unauthorized)
        {
            context.ProblemDetails.Title = "Unauthorized
                                            Access";
            context.ProblemDetails.Detail = "You are not
                    authorized to access this resource.";
        }
        else if (context.ProblemDetails.Status == StatusCodes.
                 Status404NotFound)
        {
            context.ProblemDetails.Title = "Resource Not Found";
            context.ProblemDetails.Detail = "The resource you
                              are looking for was not found.";
        }
        else
        {
            context.ProblemDetails.Title = "An unexpected error
                                            occurred";
            context.ProblemDetails.Detail = "An unexpected error
                          occurred. Please try again later.";
        }
    });

Navigate to ProductsController.cs, in the Controllers folder. Modify the endpoint that retrieves a product by its ID endpoint. We are going to specify the various responses we expect from the endpoint using the ProducesResponseType attribute and return appropriate ProblemDetails objects for error responses:

// GET: /products/{id}
[HttpGet("{id}")]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(ProductDTO))]
[ProducesResponseType(StatusCodes.Status404NotFound, Type = typeof(ProblemDetails))]
[ProducesResponseType(StatusCodes.Status401Unauthorized, Type = typeof(ProblemDetails))]
[ProducesResponseType(StatusCodes.Status500InternalServerError, Type = typeof(ProblemDetails))]
public async Task<ActionResult<ProductDTO>> GetAProduct(int id)
{
    logger.LogInformation($"Retrieving product with id {id}");

Create a try block to attempt to retrieve our product:

try
    {
        var product = await productsService.GetAProductAsync(
            id);
        if (product == null)
        {
            return Problem(
                detail: $"Product with ID {id} was not found.",
                title: "Product not found",
                statusCode: StatusCodes.Status404NotFound,
                instance: HttpContext.TraceIdentifier
                );
            }
            return Ok(product);
        }

Now add a catch for other errors. Let’s catch Unauthorized Access, which will return its own ProblemDetails:

        catch (UnauthorizedAccessException ex)
        {
            logger.LogError(ex, "Unauthorized access");
            return Problem(
                detail: ex.Message,
                title: "Unauthorized Access",
                statusCode: StatusCodes.Status401Unauthorized,
                instance: HttpContext.TraceIdentifier
            );
        }

Finally, let’s also catch general exceptions:

        catch (Exception ex)
        {
            logger.LogError(ex, $"An error occurred while                 retrieving product with id {id}");
            return Problem(
                detail: "An unexpected error occurred while                     processing your request.",
                title: "Internal Server Error",
                statusCode: StatusCodes.                    Status500InternalServerError,
                instance: HttpContext.TraceIdentifier
            );
        }
    }

Start your app with the following code:
```
dotnet run
```
Direct your web browser to go to the 404 "Not Found" URL for a ProductId that cannot exist.
Figure 1.1 illustrates trying to get an invalid ID, directly via the web browser:

Figure 1.1 – ProblemDetails returned with traceId and supportContact

How it works…

In this recipe, we relied on the built-in ProblemDetails support in ASP.NET Core 9 to create custom problem messages when your endpoints return errors.

ProblemDetails objects are automatically generated for some errors. We simply injected AddProblemDetails with the CustomizeProblemDetails class to create custom messages.

In previous versions of ASP.NET Core, we had to rely on external NuGet packages for the meaningful customization of ProblemDetails. ASP.NET Core 9 allows us to have more advanced control over the ProblemDetails response.

By customizing ProblemDetails, we can provide more detailed and useful error information to the clients, including trace IDs and support contact information.

Creating a categories endpoint using the new LINQ CountBy() method

.NET 9 introduces CountBy(), a powerful new LINQ method that simplifies the common task of grouping and counting elements. This method replaces the traditional pattern of combining GroupBy with Count, making your code more concise and readable. In this recipe, we’ll create an endpoint that uses CountBy() to efficiently report how many products exist in each category, demonstrating how this new method can simplify data aggregation tasks.

Getting ready

You can clone the starter project for this recipe here: https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/countBy.

How to do it…

In the starter project, let’s navigate to the Models folder and create a new file called CategoryDTO.cs. In this file, we will define a new DTO record:
```
namespace CountBy.Models;
public record CategoryDTO
{
    public int CategoryId { get; init; }
    public int ProductCount { get; init; }
}
```

In the Services folder, create a file named IProductsService.cs. In this file, we are going to define a contract for a GetCategoryInfoAsync service method:

using CountBy.Models;
namespace CountBy.Services;
public interface IProductsService {
    Task<IEnumerable<ProductDTO>> GetAllProductsAsync();
    Task<IReadOnlyCollection<CategoryDTO>>         GetCategoryInfoAsync();
}

Implement the service method using CountBy() on your DbContext:

public async Task<IReadOnlyCollection<CategoryDTO>> GetCategoryInfoAsync()
    {
        var products = await  context.Products.AsNoTracking().                       ToListAsync();
        var productsByCategory = products.CountBy(p =>             p.CategoryId).OrderBy(x => x.Key);
        return productsByCategory.Select(categoryGroup => new
        CategoryDTO
        {
            CategoryId = categoryGroup.Key,
            ProductCount = categoryGroup.Value
        }).ToList(
    }

Now let’s navigate to our ProductsController.cs file in the Controllers folder. Add the attributes to the CategoryInfo endpoint:

// GET: /Products/CategoryInfo
[HttpGet("CategoryInfo")]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(IEnumerable<CategoryDTO>))]
[ProducesResponseType(StatusCodes.Status204NoContent)]
[ProducesResponseType(StatusCodes.Status500InternalServerError)]

Let’s expand the GetCategoryInfo controller method:

public async Task<ActionResult<IEnumerable<CategoryDTO>>> GetCategoryInfo()
{
    logger.LogInformation("Retrieving Category Info");
    try
    {
        var products = await productsService.            GetCategoryInfoAsync();
        if (!products.Any())
            return NoContent();
        return Ok(products);
    }
    catch (Exception ex)
    {
        logger.LogError(ex, "An error occurred while                   retrieving all products");
        return StatusCode(StatusCodes.                      Status500InternalServerError);
        }
    }

Build your new project:
```
dotnet run
```
Test out your new endpoint at Products/CategoryInfo and see how many products you have in each category.
Since this is a GET endpoint, we can test it with our web browser, as shown in the following screenshot:

Figure 1.2 – Our data now with a categoryId

How it works…

We explored the use of the new LINQ CountBy() operator to create an endpoint that returns how many products you have by each category. CountBy() provides a new, more elegant way to categorize data, replacing the need to use both GroupBy() and Count() in aggregation operations. It’s important to note that CountBy() is a LINQ-to-objects method, not a LINQ-to-entities method. This means when used with EF Core, it will first materialize the query (loading all records into memory) before performing the counting operation. For large datasets in production scenarios, you might want to consider using GroupBy() directly on the IQueryable instead. In addition to database queries, CountBy() is particularly useful for in-memory operations such as analyzing API usage statistics by grouping and counting requests based on different criteria such as client IP, user agent, or endpoint path.

Implementing KeySet pagination

It is usually inadvisable to return all the available data from a GET endpoint. You may think you can get away without paging, but non-paged GET endpoints often have a surprisingly bad effect on network load and application performance. They can also prevent your API from scaling. Other resources on this topic often demonstrate OFFSET FETCH style pagination (Skip and Take when using EF Core). While this approach is easy to understand, it has a hidden cost: it forces the database engine to read through every single row leading up to the desired page.

A more efficient technique is to work only with indices and not full data rows. For ordered data, the principle is simple: if a higher ID than exists on your page can be found somewhere in the database, then you know more data is available. This is called keyset pagination.

In this recipe, we will implement keyset pagination in ASP.NET Core using EF Core, harnessing the power of indexes for optimal performance.

Getting ready

Clone the repository available here: /start/chapter01/keyset. You won’t be using any new external dependencies for this endpoint. This project has one non-paged GET endpoint.

How to do it…

In your Models folder, create an abstract base class called PagedResponse.cs:

namespace cookbook.Models;
public abstract record PagedResponse<T>
{
     public IReadOnlyCollection<T> Items { get; init; } = Array.        Empty<T>();
     public int PageSize { get; init; }
     public bool HasPreviousPage { get; init; }
     public bool HasNextPage { get; init; }
}

An important note on where to place paging logic

At this point, a lot of people would put business logic in HasPreviousPage and HasNextPage. I am not a fan of putting business logic in setters, as this tends to obfuscate the logic. It makes code harder to read as one often forgets that properties are being modified without explicit method calls. If you have to use a setter, it should handle data access and not logic. It’s a personal choice, but it is generally better to place this logic in explicit methods.

Create a PagedProductResponseDTO instance in the PagedProductResponseDTO.cs file that simply inherits from PagedResponseDTO<ProductDTO>:
```
namespace cookbook.Models;
public record PagedProductResponseDTO : PagedResponseDTO<ProductDTO>
{
}
```

Now navigate to the Services folder. Update the IProductsService interface:

using cookbook.Models;
namespace cookbook.Services;
public interface IProductsService {
    Task<IEnumerable<ProductDTO>> GetAllProductsAsync();
    Task<PagedProductResponseDTO> GetPagedProductsAsync(int     pageSize, int? lastProductId = null);
}

In the ProductsServices.cs file. Implement the GetPagedProductsAsync method. For now, you will just create a queryable on your database context:

public async Task<PagedProductResponseDTO> GetPagedProductsAsync(int pageSize, int? lastProductId = null)
    {
         var query = context.Products.AsQueryable();
     }

Before you query any data, check that an ID exists in the database that is higher than the ID of the last row you returned:

public async Task<PagedProductResponseDTO>     GetPagedProductsAsync(int pageSize, int? lastProductId =         null)
    {
         var query = context.Products.AsQueryable();
         if (lastProductId.HasValue)
        {
            query = query.Where(p => p.Id > lastProductId.                                Value);
        }

On the next line, query the remaining indexes in DbContext to get a page of products:

        var pagedProducts = await query
            .OrderBy(p => p.Id)
            .Take(pageSize)
            .Select(p => new ProductDTO
            {
                Id = p.Id,
                Name = p.Name,
                Price = p.Price,
                CategoryId = p.CategoryId
            })
            .ToListAsync();

Next, calculate the last ID from the page you just retrieved:
```
var lastId = pagedProducts.LastOrDefault()?.Id;
```

Use AnyAsync to see whether any IDs exist higher than the last one you fetched:

var hasNextPage = await context.Products.AnyAsync(
    p => p.Id > lastId);

Finish the method by returning your results along with the PageSize, HasNextPage, and HasPreviousPage metadata:

        var result = new PagedProductResponseDTO
        {
            Items = pagedProducts.Any() ? pagedProducts: Array.                        Empty<ProductDTO>(),
            PageSize = pageSize,
            HasNextPage = hasNextPage,
            HasPreviousPage = lastProductId.HasValue
        };
        return result;
    }
}

Important note

It is somewhat expensive to return a TotalCount of results. So, unless there is a clear need for the client to have a TotalCount, it is better to leave it out. You will return more robust pagination data in the next recipe.

Back in your Controller, import the built-in System.Text.Json:
```
using System.Text.Json;
```

Finally, implement a simple controller that returns your paginated data with links to both the previous page and the next page of data. First, return a bad request if no page size is given:

// GET: /Products
[HttpGet]
[ProducesResponseType(StatusCodes.Status200OK, Type = typeof(IEnumerable<ProductDTO>))]
[ProducesResponseType(StatusCodes.Status204NoContent)]
[ProducesResponseType(StatusCodes.Status500InternalServerError)]
    public async Task<ActionResult<IEnumerable<ProductDTO>>> GetProducts(int pageSize, int? lastProductId = null)
    {
        if (pageSize <= 0)
        {
            return BadRequest("pageSize must be greater than                                0");
        }

Close the method by returning a paged result:

        var pagedResult = await _productsService.            GetPagedProductsAsync(pageSize, lastProductId);
        var previousPageUrl = pagedResult.HasPreviousPage
            ? Url.Action("GetProducts", new { pageSize,
                lastProductId = pagedResult.Items.First().Id })
            : null;
        var nextPageUrl = pagedResult.HasNextPage
            ? Url.Action("GetProducts", new { pageSize,
                lastProductId = pagedResult.Items.Last().Id })
            : null;
        var paginationMetadata = new
        {
            PageSize = pagedResult.PageSize,
            HasPreviousPage = pagedResult.HasPreviousPage,
            HasNextPage = pagedResult.HasNextPage,
            PreviousPageUrl = previousPageUrl,
            NextPageUrl = nextPageUrl
        };

Finally, use Headers.Append so we don’t get yelled at for adding a duplicate header key. This could easily confuse our consuming client. We will also make sure the JSON serializer doesn’t convert our & to its Unicode character:

var options = new JsonSerializerOptions
        {
            Encoder = System.Text.Encodings.Web.
                JavaScriptEncoder.UnsafeRelaxedJsonEscaping
        };
        Response.Headers.Append("X-Pagination", 
            JsonSerializer.Serialize(
                paginationMetadata, options));
        return Ok(pagedResult.Items);

Run the app, go to http://localhost:5148/swagger/index.html, and play with your new paginator. For example, try a pageSize value of 250 and a lastProductId value of 330. Note that the metadata provides the client links to the previous and next page.
In Figure 1.3, you can see our pagination metadata being returned, via the Swagger UI:

Figure 1.3: Our pagination metadata in the x-pagination header

How it works…

We implemented a keyset paginator that works with a variable page size. Keyset pagination works with row IDs instead of offsets. When the client requests a page, the client provides both a requested page size and the ID of the last result they have consumed. This approach is more efficient than traditional skip/take pagination because it works directly with indexes rather than sorting and skipping through the entire dataset. The EF Core query behind our GetProducts endpoint avoids the more common skip/take pattern but does use the take method to retrieve the page of data. We leveraged EF Core’s AnyAsync method to directly check whether any products exist after the one fetched for the current page. We then generated URLs for the previous and next pages using Url.Action. Finally, we returned this information in a pagination metadata object to help clients navigate through the data.

Configuring a CORS policy to expose pagination metadata

In this recipe, we will allow clients to access pagination metadata from the server response by configuring a special CORS policy that exposes pagination metadata.

Getting ready

This recipe picks up exactly where the preceding recipe ended. If you are jumping around in the book, you can begin this recipe following along at https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/CORS.

How to do it…

Navigate to the Program.cs file and place the following code right after where you register your ProductService but before var app = builder.Build();:

builder.Services.AddCors(options =>
{
    options.AddPolicy("CorsPolicy", builder =>
        builder.AllowAnyOrigin()
               .AllowAnyMethod()
               .AllowAnyHeader()
               .WithExposedHeaders("X-Pagination"));
});

Right before app.MapControllers(), enable the CORS policy, like so:
```
app.UseCors("CorsPolicy");
app.MapControllers();
app.Run();
```
Run your API with the new CORS policy:
```
dotnet run
```
One way to confirm that CORS is allowing our response headers to be displayed is simply via the Network tab in our browser:

Figure 1.4: Note HasPreviousPage and HasNextPage in the X-Pagination header

Important note

Keep in mind that, when testing on localhost, a CORS policy is more lenient and you will probably see these headers regardless. You might not see the full impact of CORS during local development. This recipe is critical when deploying your web API and allowing a variety of clients to consume your API.

How it works…

We applied a CORS policy that allows requests from any origin, AllowAnyOrigin. When the client consuming our API is hosted on a different origin than the API, we have to start thinking about CORS policies. We added the WithExposedHeaders("X-Pagination") policy to ensure that the header that contains our pagination data is accessible to the client.

Implementing efficient first- and last-page access with EF Core

In this recipe, we’ll expand our keyset pagination implementation to efficiently handle first and last page access by leveraging EF Core’s entity tracking and Find method. Users often navigate directly to the first or last page of paginated results, so these pages should load as quickly as possible, while still remaining reasonably fresh.

Getting ready

This recipe builds on the two preceding recipes. You can clone the starter project here: https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/firstLastPage.

How to do it…

Open the Program.cs file. Register an in-memory cache on the line after AddControllers();:
```
builder.Services.AddMemoryCache();
```

Open the PagedResponse.cs file inside the Models folder. Update your PagedResponse model to include TotalPages:

namespace cookbook.Models;
public abstract record PagedResponse<T>
{
     public IReadOnlyCollection<T> Items { get; init; } = Array.        Empty<T>();
     public int PageSize { get; init; }
     public bool HasPreviousPage { get; init; }
     public bool HasNextPage { get; init; }
     public int TotalPages { get; init; }
}

Open ProductReadService.cs in the Services folder. At the bottom of the class, create a new helper method for retrieving and caching total pages. When it is time to recalculate the total pages count, we are going to take that opportunity to clear EF Core’s change tracker—forcing a fresh first and last page:

public async Task<int> GetTotalPagesAsync(int pageSize)
{
    if (!cache.TryGetValue(TotalPagesKey, out int totalPages))
    {
        context.ChangeTracker.Clear();
        var totalCount = await context.Products.CountAsync();
        totalPages = (int)Math.Ceiling(totalCount / (double)                      pageSize);
        cache.Set(TotalPagesKey, totalPages, 
            TimeSpan.FromMinutes(2));
    }
    return totalPages;
}

Important note

We have used a basic ResponseCache in the controller previously, but this is the first time we are introducing caching to the service layer.

On the next line, create another very simple helper method for invalidating the cached total pages:
```
public void InvalidateCache()
{
    Cache.Remove(TotalPagesKey);
}
```

Still in the ProductReadService.cs file, scroll up to the top of the file, and add the constant for our cached TotalPages key at the top of the ProductReadService class, after the class definition:

using Microsoft.Extensions.Caching.Memory;
public class ProductReadService(AppDbContext context, IMemoryCache cache) : IProductReadService
{
    private const string TotalPagesKey = "TotalPages";

Still in the ProductReadService.cs file, delete the entire GetPagedProductsAsync method implementation. We’ll rebuild it to leverage EF Core’s entity tracking and Find method.

Continuing in ProductReadService.cs, let’s start rebuilding GetPagedProductsAsyncMethod. Start with the method signature and variables we will need:

public async Task<PagedProductResponseDTO> GetPagedProductsAsync(int pageSize, int? lastProductId = null)
{
    var totalPages = await GetTotalPagesAsync(pageSize);
    List<Product> products;
    bool hasNextPage;
    bool hasPreviousPage;

On the next line, add the first-page handling logic using Find:

if (lastProductId == null)
{
    products = new List<Product>();
    for (var i = 1; i <= pageSize; i++)
    {
        var product = await context.Products.FindAsync(i);
        if (product != null)
        {
            products.Add(product);
        }
    }
    hasNextPage = products.Count == pageSize;
    hasPreviousPage = false;
}

On the next line, add the last-page handling logic:

else if (lastProductId == ((totalPages - 1) * pageSize))
{
    products = new List<Product>();
    for (var i = lastProductId.Value; i < lastProductId.Value +          pageSize; i++)
    {
        var product = await context.Products.FindAsync(i);
        if (product != null)
        {
            products.Add(product);
        }
    }
    hasNextPage = false;
    hasPreviousPage = true;
}

Now, before we place our regular keyset pagination logic, let’s take this opportunity to clear the ChangeTracker so a fresh first and last pages will be returned. On the next line, place this:
```
else
{
    context.ChangeTracker.Clear();
```

On the next line, let’s implement our ordinary keyset pagination logic. Note: it is critical that we do not use AsNoTracking() in our query:

    IQueryable<Product> query = context.Products;
    query = query.Where(p => p.Id > lastProductId.Value);
    products = await query
        .OrderBy(p => p.Id)
        .Take(pageSize)
        .ToListAsync();
    var lastId = products.LastOrDefault()?.Id;
    hasNextPage = lastId.HasValue &&
        await context.Products.AnyAsync(p => p.Id > lastId);
    hasPreviousPage = true;
}

Add the return statement and close the GetPagedProductsAsync method:

return new PagedProductResponseDTO
    {
        Items = products.Select(p => new ProductDTO
        {
            Id = p.Id,
            Name = p.Name,
            Price = p.Price,
            CategoryId = p.CategoryId
        }).ToList(),
        PageSize = pageSize,
        HasPreviousPage = hasPreviousPage,
        HasNextPage = hasNextPage,
        TotalPages = totalPages
    };
}

Finally, open the ProductsController.cs file in the Controller folder. Let’s modify the pagination in the GetProducts action method to include FirstPageUrl and LastPageUrl after NextPageUrl:

var paginationMetadata = new
{
    PageSize = pagedResult.PageSize,
    HasPreviousPage = pagedResult.HasPreviousPage,
    HasNextPage = pagedResult.HasNextPage,
    TotalPages = pagedResult.TotalPages,
    PreviousPageUrl = pagedResult.HasPreviousPage
        ? Url.Action("GetProducts", new { pageSize,         lastProductId = pagedResult.Items.First().Id })
        : null,
    NextPageUrl = pagedResult.HasNextPage
        ? Url.Action("GetProducts", new { pageSize,         lastProductId = pagedResult.Items.Last().Id })
        : null,
    FirstPageUrl = Url.Action("GetProducts", new { pageSize }),
    LastPageUrl = Url.Action("GetProducts", new { pageSize,         lastProductId = (pagedResult.TotalPages - 1) * pageSize     })
    };
    // method continues

Run the web API:
```
dotnet run
```
Open your web browser and navigate to the Swagger UI interface at http://localhost:<yourport>/swagger/index.html. Try the Products endpoint. Note the first- and last-page URLs in the X-Pagination header as shown in the following screenshot:

Figure 1.5 – FirstPageUrl and LastPageUrl

To navigate to the last page, try entering the page size and product ID into the Swagger boxes representing query parameters. If you are using a debugger, you’ll see Find retrieving products from the change tracker without hitting the database.

How it works…

This recipe leverages EF Core’s entity tracking system and Find method to efficiently serve the first and last page. We used IMemoryCache to cache only the total page calculation. We did not use IMemoryCache to cache the actual product data (which is the approach we would take with output caching). Instead, we let EF Core’s change tracker handle entity caching through Find. Note that Find will not execute a database query if the entity is already loaded into the change tracker. To prevent stale data, we clear the change tracker at two strategic points: during regular pagination and when recalculating the total page count every two minutes. This dual invalidation strategy ensures that while the first and last pages can be served quickly from the tracker, no tracked entity can be stale for more than two minutes. Since the total count typically changes less frequently than individual records, the total count is a better candidate for formal caching in IMemoryCache compared to caching the entire result set.

Testing the API in PowerShell

When testing your API, relying solely on tools such as cURL or Swagger can limit your options. In this recipe, we will confirm that the API is correctly implemented via PowerShell. We will retrieve pagination metadata, navigate through pages, and ensure that the returned data is what we expect it to be. We will accomplish this using the built-in Invoke-WebRequest to inspect our custom X-Pagination header. We will type this recipe directly into the PowerShell terminal.

Important note

While we’ll be using PowerShell in this recipe, rest assured that we’ll also cover other popular tools such as Postman throughout this book—equipping you with a diverse set of testing techniques.

Getting ready

Clone the starting code from here: https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/psTesting. It contains a web API similar to the other APIs we have built in this chapter.

You will need to open PowerShell or use something like Windows Terminal with PowerShell. PowerShell is cross-platform, so you don’t have to be on Windows to follow along with this recipe. The end project folder for this recipe has these commands saved in a PowerShell script if you want to compare. Instead of writing a script file, we will be entering these commands directly into the terminal.

How to do it…

Run the application using the following command:
```
dotnet run
```
Open your PowerShell terminal. Let’s create some variables for your test URL. Save your baseUrl and the endpoint you want to test with pageSize in separate variables:
```
$baseUrl = "http://localhost:5148"
$testEndpoint = "/Products?pageSize=10"
$fullUrl = $baseUrl + $testEndpoint;
```

Important note

Remember: your baseUrl has whatever port number dotnet run is serving your API on.

In the terminal, you should now see our API being served on the local host.

Figure 1.6 – dotnet run starting the API on port 5148, your port may be different

Now call the endpoint and save the response in a variable:

$response = Invoke-WebRequest -Uri $fullUrl -Headers @{"Accept" = "application/json"}

Important note

One alternative way of querying an endpoint in PowerShell is Invoke-RestMethod, which directly converts a JSON response to a PowerShell object. However, Invoke-WebRequest has advantages, as it provides access to more detailed information about the HTTP response, including headers, status codes, and cookies.

Save the pagination metadata in variables:

$xPaginationHeader = $response.Headers["X-Pagination"]
$xPagination = $xPaginationHeader | ConvertFrom-Json

Ensure that the correct pagination data is present.
Figure 1.7 shows the $xPagination variable in our PowerShell terminal:

Figure 1.7 – Pagination metadata displayed in PowerShell

Save a URL for NextPage in a variable:

$nextPageUrl = $baseUrl + $xPagination.NextPageUrl

Call the next page and examine the results:

$response = Invoke-WebRequest -Uri $nextPageUrl
$jsonContent = $response.Content | ConvertFrom-Json
$jsonContent | Format-Table -AutoSize

Figure 1.8 shows the next page of data displayed in PowerShell:

Figure 1.8 – The next page of data displayed in PowerShell

How it works…

We verified our pagination headers and played around with paged data directly in the terminal. We also formatted response content in PowerShell and learned about Invoke-WebRequest, which is one way we can manipulate our APIs from the terminal. The response from Invoke-WebRequest includes headers, which we accessed to get the X-Pagination custom header. This header contains our pagination metadata such as the total number of pages, the current page, and links to the next and previous page. Invoke-WebRequest lets you access all response headers directly, making it easy to parse a custom header like X-Pagination.

Another similar cmdlet is Invoke-RestMethod, which automatically parses your JSON and returns a smaller response object. The advantage of Invoke-WebRequest is that it can include more information. Invoke-RestMethod is fantastic for simple REST API interactions but Invoke-WebRequest can be better suited for complex interactions.

Using the new AggregateBy LINQ method to return the average price per category

In this recipe, we will demonstrate a practical example of using the new AggregateBy LINQ method. We are going to return the average price per product on each page our API returns using AggregateBy. Now, AggregateBy is one of the three new LINQ methods in .NET 9. It is another grouping method like CountBy but with a twist: you provide your own seed and aggregation function.

Getting ready

In this recipe, we will build on our previous paged GET endpoint that returns links to previous and next pages.

You can clone the starting project here: https://github.com/PacktPublishing/ASP.NET-9-Web-API-Cookbook/tree/main/start/chapter01/aggregateBy.

How to do it…

Add a dictionary to the existing PagedResponse abstract record in Models/PagedResponse.cs. This will hold your average price per category:

    public Dictionary<int, decimal> AveragePricePerCategory
    { get; init; } = new();
    }

Your record should now look like this:

namespace AggregateBy.Models;
public abstract record PagedResponse<T>
{
     public IReadOnlyCollection<T> Items { get; init; } = Array.        Empty<T>();
     public int PageSize { get; init; }
     public bool HasPreviousPage { get; init; }
     public bool HasNextPage { get; init; }
     public int TotalPages { get; init; }
     public int LastPage { get; init; }
     public Dictionary<int, decimal> AveragePricePerCategory {         get; init; } = new();
}

In your service, create a private helper method to calculate your average price per category using AggregateBy:

private async Task<Dictionary<int, decimal>> GetAveragePricePerCategoryAsync(List<ProductDTO> products)
    {
        if (products == null || !products.Any())
        {
            return new Dictionary<int, decimal>();
        }
        var aggregateByTask = Task.Run(() =>
        {
            var aggregateBy = products.AggregateBy(
                product => product.CategoryId,
                x => (Sum: 0m, Count: 0),
                (acc, product) => (acc.Sum + product.Price, acc.                                   Count + 1)
            );
            var averagePriceByCategory = aggregateBy.            ToDictionary(
                kvp => kvp.Key,
                kvp => Math.Round(kvp.Value.Sum / kvp.Value.                                  Count, 2)
            );
            return averagePriceByCategory;
        });
        return await aggregateByTask;
    }

Await the helper method and add the aggregated data to the response:

        var averagePricePerCategory = await GetAveragePricePerCategoryAsync(pagedProducts);
        var result = new PagedProductResponseDTO
        {
            Items = pagedProducts,
            PageSize = pageSize,
            HasNextPage = hasNextPage,
            HasPreviousPage = lastProductId.HasValue,
            AveragePricePerCategory = averagePricePerCategory
        };
        return result;

In ProductsController.cs, update the returned metadata object to include AveragePricePerCategory:

        var paginationMetadata = new
        {
            PageSize = pagedResult.PageSize,
            HasPreviousPage = pagedResult.HasPreviousPage,
            HasNextPage = pagedResult.HasNextPage,
            PreviousPageUrl = previousPageUrl,
            NextPageUrl = nextPageUrl,
            AveragePricePerCategory = pagedResult.                AveragePricePerCategory
        };

Run the new version of your web API and observe the new metadata (either by using CURL, by using your browser’s network tab, by navigating to swagger UI http:localhost:<yourport>/swagger/index.html, or by using PowerShell’s Invoke-WebRequest):

x-pagination: {"PageSize":400,"HasPreviousPage":true,"HasNextPage":true,"PreviousPageUrl":"/Products?pageSize=400 &lastProductId=81","NextPageUrl":"/Products?pageSize=400 &lastProductId=480","AveragePricePerCategory":{"3":1066.73,"1":937.33,"4":1038.46,"2":956.31,"5":1121.53}}

How it works…

In this recipe, we learned how to take our returned list of objects and calculate additional category grouping information using the new LINQ method AggregateBy. Now, AggregateBy can be a bit confusing, so let’s break it down:

var aggregateBy = products.AggregateBy(
                product => product.CategoryId,
                x => (Sum: 0m, Count: 0),
                (acc, product) => (acc.Sum + product.Price, acc.Count                                                             + 1)

We first pass the property we want to group by product => product.CategoryId. Then, we pass a tuple of seed values (Sum: 0m, Count: 0). Note that 0m makes sure that sum starts at 0 as a decimal.

AggregateBy goes to work on that tuple and returns a tuple of the newly aggregated products by category. The resulting tuple is then converted to a dictionary while calculating the average price.

This is all done inside Task.Run to ensure the aggregation runs asynchronously.

ASP.NET Core 9 Web API Cookbook: Over 60 hands-on recipes for building and securing enterprise web APIs with REST, GraphQL, and more

What do you get with eBook?

Contact Details

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with eBook?

Contact Details

Billing Address

Product Details

Packt Subscriptions

Table of Contents

Recommendations for you

About the authors

FAQs