Published
- 5 min read
Integrating ElasticSearch with .NET Web API
Integrating ElasticSearch with .NET Web API
ElasticSearch offers several benefits, including high-performance search, real-time search, full-text search, faceting, geolocation search, analytics capabilities, ease of use, scalability, reliability, and open-source nature. These features make it a popular choice for search and analytics applications, as it can handle large datasets, provide fast and accurate results, and be easily integrated into different systems.
Setting up ElasticSearch Locally using Docker Compose
Let’s begin by creating a new .NET project and adding a new file named docker-compose.yml to the root of the project. This file will define the services used in the Docker Compose environment.
version: '3.8'
services:
elasticsearch:
container_name: els
image: elasticsearch:8.15.0
ports:
- "9200:9200"
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data
environment:
- discovery.type=single-node
- xpack.security.enabled=false
networks:
- elk
kibana:
container_name: kibana
image: kibana:8.15.0
ports:
- "5601:5601"
depends_on:
- elasticsearch
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
networks:
- elk
networks:
elk:
driver: bridge
volumes:
elasticsearch-data:
This docker-compose.yml file defines two services: Elasticsearch and Kibana.
The Elasticsearch service is configured to run a single node of Elasticsearch version 8.15.0. It maps port 9200 of the container to port 9200 of the host machine. The volumes section defines a volume named elasticsearch-data that will store the data of the Elasticsearch service. The environment section disables Elasticsearch’s security features.
The Kibana service is configured to run Kibana version 8.15.0. It maps port 5601 of the container to port 5601 of the host machine. Kibana is a web interface for Elasticsearch that allows you to visualize and interact with your data.
To start the Elasticsearch and Kibana services, open a terminal window and run the following command:
docker-compose up
This command will start the Elasticsearch and Kibana services in the background. Once the services are running, you can access Kibana by navigating to http://localhost:5601 in your web browser.
Integrating ElasticSearch with a .NET Web API
Create a new .NET Web API project and install the following NuGet package:
dotnet add package Elastic.Clients.Elasticsearch
This package provides the necessary classes to interact with Elasticsearch from .NET.
Next, create a new folder named Models and add a new class named User:
public class User
{
[JsonProperty("Id")]
public int Id { get; set; }
[JsonProperty("FirstName")]
public string FirstName { get; set; }
[JsonProperty("LastName")]
public string LastName { get; set; }
}
Then create a new folder named Services and add a new interface named IElasticService:
public interface IElasticService
{
Task CreateIndexIfNotExistsAsync(string indexName);
Task<bool> AddOrUpdate(User user);
Task<bool> AddOrUpdateBulk(IEnumerable<User> users, string indexName);
Task<User> Get(string key);
Task<List<User>> GetAll();
Task<bool> Remove(string key);
Task<long> RemoveAll();
}
Add a new class named ElasticService to the Services folder to implement the IElasticService interface:
public class ElasticService : IElasticService
{
private readonly ElasticsearchClient _client;
private readonly ElasticSettings _elasticSettings;
public ElasticService(IOptions<ElasticSettings> options)
{
_elasticSettings = options.Value;
var settings = new ElasticsearchClientSettings(new Uri(_elasticSettings.Url))
.DefaultIndex(_elasticSettings.DefaultIndex);
_client = new ElasticsearchClient(settings);
}
public async Task CreateIndexIfNotExistsAsync(string indexName)
{
if (!_client.Indices.Exists(indexName).Exists)
{
await _client.Indices.CreateAsync(indexName);
}
}
public async Task<bool> AddOrUpdate(User user)
{
var response = await _client.IndexAsync(user, idx => idx
.Index(_elasticSettings.DefaultIndex)
.Id(user.Id)
.Refresh(Refresh.WaitFor));
return response.IsValidResponse;
}
public async Task<bool> AddOrUpdateBulk(IEnumerable<User> users, string indexName)
{
var response = await _client.BulkAsync(b => b
.Index(_elasticSettings.DefaultIndex)
.UpdateMany(users, (ud, u) => ud.Doc(u).DocAsUpsert(true)));
return response.IsValidResponse;
}
public async Task<User> Get(string key)
{
var response = await _client.GetAsync<User>(key,
g => g.Index(_elasticSettings.DefaultIndex));
return response.Source;
}
public async Task<List<User>> GetAll()
{
var response = await _client.SearchAsync<User>(s => s
.Index(_elasticSettings.DefaultIndex));
return response.IsValidResponse ? response.Documents.ToList() : default;
}
public async Task<bool> Remove(string key)
{
var response = await _client.DeleteAsync<User>(key,
d => d.Index(_elasticSettings.DefaultIndex));
return response.IsValidResponse;
}
public async Task<long> RemoveAll()
{
var response = await _client.DeleteByQueryAsync<User>(d => d
.Index(_elasticSettings.DefaultIndex)
.Query(q => q.MatchAll()));
return response.IsValidResponse ? response.Deleted : default;
}
}
Configuring the .NET Web API to Use ElasticSearch
Open the appsettings.json file and add the following section:
"ElasticSettings": {
"Url": "http://localhost:9200",
"DefaultIndex": "users",
"Username": "",
"Password": ""
}
Open the Program.cs file and add the following code:
builder.Services.Configure<ElasticSettings>(builder.Configuration.GetSection("ElasticSettings"));
builder.Services.AddSingleton<IElasticService, ElasticService>();
Creating an ElasticSearch Controller
Create a new controller named UsersController:
[ApiController]
[Route("[controller]")]
public class UsersController : ControllerBase
{
private readonly ILogger<UsersController> _logger;
private readonly IElasticService _elasticService;
public UsersController(ILogger<UsersController> logger, IElasticService elasticService)
{
_logger = logger;
_elasticService = elasticService;
}
[HttpPost("create-index")]
public async Task<IActionResult> CreateIndex(string indexName)
{
await _elasticService.CreateIndexIfNotExistsAsync(indexName);
return Ok($"Index {indexName} created or already exists.");
}
[HttpPost("add-user")]
public async Task<IActionResult> AddUser([FromBody] User user)
{
var result = await _elasticService.AddOrUpdate(user);
return result ? Ok("User added or updated successfully.") : StatusCode(500, "Error adding or updating user.");
}
[HttpPost("update-user")]
public async Task<IActionResult> UpdateUser([FromBody] User user)
{
var result = await _elasticService.AddOrUpdate(user);
return result ? Ok("User updated successfully.") : StatusCode(500, "Error updating user.");
}
[HttpGet("get-user/{key}")]
public async Task<IActionResult> GetUser(string key)
{
var user = await _elasticService.Get(key);
return user != null ? Ok(user) : NotFound("User not found.");
}
[HttpGet("get-all-users")]
public async Task<IActionResult> GetAllUsers()
{
var users = await _elasticService.GetAll();
return users != null ? Ok(users) : StatusCode(500, "Error retrieving users.");
}
[HttpDelete("delete-user/{key}")]
public async Task<IActionResult> DeleteUser(string key)
{
var result = await _elasticService.Remove(key);
return result ? Ok("User deleted successfully.") : StatusCode(500, "Error deleting user.");
}
}
Conclusion
We’ve explored the process of integrating ElasticSearch with a .NET Web API. We learned how to set up ElasticSearch and Kibana locally using Docker Compose, connect to ElasticSearch from the web API, and create simple CRUD operations.
ElasticSearch offers several benefits, such as high-performance search, real-time search, full-text search, faceting, geolocation search, analytics capabilities, ease of use, scalability, reliability, and open-source nature. By leveraging these benefits, developers can build powerful and efficient search and analytics applications.