Hey Friends ,
today we are going to do leverage Application Insights (AI) capabilities, and will add our custom properties to be logged with the AI request telemetry.
you will need an AI resource to configure our app with.
In this scenario we are going to log Headers and Body (Payload) for each and every request and response our application process.
From a Design perspective some people say that it`s not recommended to have such logging mechanism running on a production environment, and you may need it only for development and issues tracking no more.
my opinion that it depends on your solution, your data, conditions and polices you are running with.
So We are going to add a toggling option to enable/disable the logging middle-ware, also logging toggle for each property. to make the it more dynamic. Let`s Start
- Create a .Net Core Web Api App.
- let`s make a simple update with the default Values Controller to look like that
[Route(“api/[controller]”)]
[ApiController]
public class ValuesController : ControllerBase
{
// POST api/values
[HttpPost]
public ActionResult<AddNumbersRespone> Post([FromBody] AddNumbersRequest bodyBag)
{
Response.Headers.Add(“XX-ServerRespondTime”, DateTime.Now.ToShortTimeString());
return (new AddNumbersRespone() { Result = bodyBag.Num1 + bodyBag.Num2 });
}
}
public class AddNumbersRespone
{
public int Result { get; set; }
}public class AddNumbersRequest
{
public int Num1 { get; set; }
public int Num2 { get; set; }
}
Response.Headers.Add(“XX-ServerRespondTime”, DateTime.Now.ToShortTimeString());
This line just add a simple header to Response headers list for Testing purpose.
- Test This simple Api
PostMan Call To the api
Now Lets Add and configure Application Insights with the solution
right click to the solution and configure Application Insights. if you are not familir with application Insights find it Here .
let`s add an appsettings object to represent our logging toggle
“AppInsightsCustomMiddlewareSettings”: {
“IsEnabled”: true,
“RequestHeaders”: true,
“RequestBody”: true,
“ResponseHeaders”: true,
“ResponseBody”: true
}
for better reading and working with the AppInsightsCustomMiddlewareSettings conf i am going to add a class to bind the settings to.
- add new simple class AppInsightsCustomMiddlewareSettings add it under a new Helper Folder
public class AppInsightsCustomMiddlewareSettings
{
public bool IsEnabled { get; set; }
public bool RequestHeaders { get; set; }
public bool RequestBody { get; set; }
public bool ResponseHeaders { get; set; }
public bool ResponseBody { get; set; }
}
- Add newClass AppInsightsCutomProperties to represent our Middle-ware
Now Lets Add our Middleware assets to the class
private readonly RequestDelegate _next;
private readonly AppInsightsCustomMiddlewareSettings _AppInsightsConf;
readonly IConfiguration _configuration;public AppInsightsCutomProperties(RequestDelegate next, IConfiguration configuration)
{
_next = next;
_configuration = configuration;
_AppInsightsConf= new AppInsightsCustomMiddlewareSettings();
_configuration.Bind(“AppInsightsCustomMiddlewareSettings”, _AppInsightsConf);
}
public async Task InvokeAsync(HttpContext context)
{
await _next(context);
}
here we add the RequestDelegate next to have access to the middle ware pipeline. also we injected a configuration object to read our settings values and bind it to _AppInsightsconf object.
and Here is our MiddleWare InvokeAsync Task function.
for now it dose no thing just passes the Httpcontext value to the next middle-ware.
if you are not familiar with how .Net-Core pipeline middle-ware works take a look Here
also we added
public static class UseAppInsightsCutomPropertiesMiddlewareExtensions
{
public static IApplicationBuilder UseAppInsightsCutomProperties(this IApplicationBuilder builder)
{
return builder.UseMiddleware<AppInsightsCutomProperties>();
}
}
as an Extension to allow our middle ware to be injected into the Pipeline of our app.
it should look like this now :
- Now Lets get an Object to represent the current (AI) requestTelemetry by adding in our Middle-Ware Invoke function
RequestTelemetry requestTelemetry = context.Features.Get<RequestTelemetry>();
with using Microsoft.ApplicationInsights.DataContracts;
lets start do our logging stuff Now.
- Logging the Request and Response Payload is a bit tricky. we will need to map them into a memory stream to be able to read and seek on them. first we will log the request Body then Make the call into the next middle-wares later we will log the response body and dispose the streams back.
Stream RequestStream=null,ResponseStream=null;
MemoryStream RequestBodyMemoryBuffer = null, ResponseBodyMemoryBuffer=null;//Request Body Pre Actions
if (_conf.RequestBody)
{
RequestStream = context.Request.Body;
RequestBodyMemoryBuffer = new MemoryStream();
// Copy the request stream to the memory stream.
await RequestStream.CopyToAsync(RequestBodyMemoryBuffer);
RequestBodyMemoryBuffer.Position = 0L;
requestTelemetry.Properties.Add(“RequestBody”, Encoding.ASCII.GetString(RequestBodyMemoryBuffer.ToArray()));
context.Request.Body = RequestBodyMemoryBuffer;
}//Response Body Pre Actions
if (_conf.ResponseBody) {
ResponseStream = context.Response.Body;
ResponseBodyMemoryBuffer = new MemoryStream();
context.Response.Body = ResponseBodyMemoryBuffer;
}await _next(context);
also we need to check and log request headers so lets add check if The RequestHeaders log is enabled and log it to our telemetry before we call await _next(context);
if (_AppInsightsConf.RequestHeaders)
{
foreach (var header in context.Request.Headers)
{
if(header.Key.ToLower().StartsWith(“XX-“))
requestTelemetry.Properties.Add($”RequestHeader-{header.Key}”, header.Value);
}
}
here we get only our buisness headers that we sent, starts with “XX-” to avoid logging useless headers.
2.we have logged our request headers and body . let the request pipeline continue to the rest of middlewares by this line await _next(context);
After that we need to log the response body, and Dispose back the memory streams we used . to look like that.
[….]
await _next(context);//Response Body Post Action
if (_conf.ResponseBody) {
context.Response.Body.Seek(0, SeekOrigin.Begin);
string BodyStr= await new StreamReader(context.Response.Body).ReadToEndAsync();
requestTelemetry.Properties.Add(“ResponseBody”, BodyStr);
context.Response.Body.Seek(0, SeekOrigin.Begin);
await ResponseBodyMemoryBuffer.CopyToAsync(ResponseStream);
ResponseBodyMemoryBuffer.Dispose();
}//Request Body Post Actions
if (_conf.RequestBody)
{
context.Request.Body = RequestStream;
RequestBodyMemoryBuffer.Dispose();
}
- lets check ResponseHeaders is enabled and log it.
if (_AppInsightsConf.ResponseHeaders)
{
foreach (var header in context.Response.Headers)
{
if (header.Key.ToLower().StartsWith(“XX-“) && !requestTelemetry.Properties.ContainsKey(header.Key))
requestTelemetry.Properties.Add($”ResponseHeader-{header.Key}”, header.Value);
}
}
the final Invoke Method should look like that.
public async Task InvokeAsync(HttpContext context)
{
RequestTelemetry requestTelemetry = context.Features.Get<RequestTelemetry>();
Stream RequestStream = null, ResponseStream = null;
MemoryStream RequestBodyMemoryBuffer = null, ResponseBodyMemoryBuffer = null;//Request Body Pre Actions
if (_AppInsightsConf.RequestBody)
{
RequestStream = context.Request.Body;
RequestBodyMemoryBuffer = new MemoryStream();
// Copy the request stream to the memory stream.
await RequestStream.CopyToAsync(RequestBodyMemoryBuffer);
RequestBodyMemoryBuffer.Position = 0L;
requestTelemetry.Properties.Add(“RequestBody”, Encoding.ASCII.GetString(RequestBodyMemoryBuffer.ToArray()));
context.Request.Body = RequestBodyMemoryBuffer;
}//Response Body Pre Actions
if (_AppInsightsConf.ResponseBody)
{
ResponseStream = context.Response.Body;
ResponseBodyMemoryBuffer = new MemoryStream();
context.Response.Body = ResponseBodyMemoryBuffer;
}//Read request business Headers
if (_AppInsightsConf.RequestHeaders)
{
foreach (var header in context.Request.Headers)
{
if (header.StartsWith(“XX-“))
requestTelemetry.Properties.Add($”RequestHeader-{header.Key}”, header.Value);
}
}await _next(context);
//Response Body Post Action
if (_AppInsightsConf.ResponseBody)
{
context.Response.Body.Seek(0, SeekOrigin.Begin);
string BodyStr = await new StreamReader(context.Response.Body).ReadToEndAsync();
requestTelemetry.Properties.Add(“ResponseBody”, BodyStr);
context.Response.Body.Seek(0, SeekOrigin.Begin);
await ResponseBodyMemoryBuffer.CopyToAsync(ResponseStream);
ResponseBodyMemoryBuffer.Dispose();
}//Request Body Post Actions
if (_AppInsightsConf.RequestBody)
{
context.Request.Body = RequestStream;
RequestBodyMemoryBuffer.Dispose();
}//Read response business Headers
if (_AppInsightsConf.ResponseHeaders)
{
foreach (var header in context.Response.Headers)
{
if (header.Key.StartsWith(“XX-“) && !requestTelemetry.Properties.ContainsKey(header.Key))
requestTelemetry.Properties.Add($”ResponseHeader-{header.Key}”, header.Value);
}
}
}
now our middle ware Functionality is ready. BUT we didn’t injected it into the application pipeline Yet.
So head to startup.cs in Configure method. and lets add our line
app.UseAppInsightsCutomProperties();
before
app.UseHttpsRedirection();
app.UseMvc();
but we need also to add it if the IsEnabled toggle is true. So get the settings here check for that the final configure service should look like that.
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}var logConf = new AppInsightsCustomMiddlewareSettings();
Configuration.Bind(“AppInsightsCustomMiddlewareSettings”, logConf);
if (logConf.IsEnabled)
{
app.UseAppInsightsCutomProperties();
}app.UseHttpsRedirection();
app.UseMvc();
}
Now Our App is ready to run.
run the app and make a request using post man
go open the application insights Screen
- Select Time range to be last 30 min
- select the Post request
- click request details
you can see our custom properties appears here .
RequestBody and Request-XX-tra**
ResponseBody and Response-XX-ser***
you can also check the toggling mechanism. go set RequsetBody and ResponseBody to false in the appsettings. then make a new request call
Also You can disable the whole middle ware not to be added to application pipeline using IsEnabled in app settings.
It`s Done.
You Can Check The Whole Solution Code Here at GitHub
Thanks
Thoughts, for a better Software. 