DRNJ

The Problem

I made the mistake of trying to install Ubuntu on an SSD on an already working Raspberry Pi 4. The device as Raspian on it and was working normally for over six months. However, I was playing with docker and wanted some features not available in Raspbian. Rather than flash an SD Card I thought that I would use a Samsung 840 Pro 500GB SSD that I had lying around.

I followed the instructions from Ubuntu I tried to install Ubuntu Server 21 64Bit. The SSD was written correctly but when I booted the Pi 4 the boot took ages and I got many messages of the type

EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #525023

on the console. I tried Ubuntu 20 instead, this would not boot. I tried workstation 21 and it showed the same issues as server. So I reverted to Ubuntu 21 server.

Investigation

I searched for P i4 Ubuntu SSD issues and found this article. So at least I was not alone in my problems.

After more searching I found an article that suggested adding

pcie_aspm=force

or

nvme_core.default_ps_max_latency_us=5500

To the grub configuration.

Great…apart from it seems that Ubuntu on a Pi 4 doesn’t seem to use grub (perhaps I am wrong) so I couldn’t update any grub boot configuration.

A bit of searching at turns out that on the Pi you can use “quirks” mode for the USB controller. All you need is the device ID….

Following this article I found out how to get this ID for my USB enclosure.

The Solution

I flashed the SSD with a clean install of Ubuntu Server 21 (on my Windows 10 desktop). The SSD has a boot device which is readble from WIndows. I edited cmdline.txt and added

usb-storage.quirks=152d:0578:u

to the end of the single line in that file

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc quiet splash usb-storage.quirks=152d:0578:u

The 152d:0578 is the ID of my USB controller.

Plug the SSD into the Pi 4 and, voila, it boots and works

The Problem

In .NET 4.x it was straightforward to read the request body of an incoming HTTP request, log the information and also log the outgoing response back to the caller. I thought it would be just as easy in .Net Core, how wrong a I was as Microsoft do their utmost to hide the Request body from you.

The Solution

Accessing the Request and Response bodies must be done in HTTP Middleware, and, after reading much on StackOverflow I came up with the following solution (and my thanks to all the original authors and people who helped me). I cannot take credit for this code, I just adapted it from ideas on StackOverflow

Remember 

For .Net Core 2.1  use  context.HttpContext.Request.EnableRewind();
For .Net Core 3.1  context.HttpContext.Request.EnableBuffering();

/// <summary>
/// Want to be able to read the Request.Body to get the incoming JSON
/// In .net 4 this was more or less simple. In Core it has changed
/// Need to be able to "rewind" of Request.Body - have to do that
/// Thanks to
/// https://stackoverflow.com/questions/40494913/how-to-read-request-body-in-a-asp-net-core-webapi-controller
/// in request pipeline middleware
/// 
/// For ASP.NET 2.1
/// context.HttpContext.Request.EnableRewind();
/// For ASP.NET 3.1
/// context.HttpContext.Request.EnableBuffering();
///
/// See also https://stackoverflow.com/questions/43403941/how-to-read-asp-net-core-response-body/43404745
/// </summary>
public class LogRequestResponseMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger _logger;

private readonly RecyclableMemoryStreamManager _recyclableMemoryStreamManager;
private const int ReadChunkBufferLength = 4096;

private bool LogRequest;
private bool logResponse;

public LogRequestResponseMiddleware(RequestDelegate next, ILoggerFactory loggerFactory, bool requestLogging, bool responseLogging)
{
//--------------------------------------------------------------------
// Config |
//--------------------------------------------------------------------

this.LogRequest = requestLogging;
this.logResponse = responseLogging;

_next = next;
_logger = loggerFactory.CreateLogger<LogRequestResponseMiddleware>();

//--------------------------------------------------------------------
// Create stream to use in place of HttpContext.Response.Body stream |
// which is write-only |
//--------------------------------------------------------------------

_recyclableMemoryStreamManager = new RecyclableMemoryStreamManager();
}


public virtual async Task Invoke(HttpContext context)
{
//-----------------------------------------------------
// Request |
//-----------------------------------------------------

if (this.LogRequest)
{
//-------------------------------------------------
// Turn on Buffering/Rewind |
//-------------------------------------------------
context.Request.EnableBuffering();

//-----------------------------------------------------
// Log Request |
//-----------------------------------------------------
await this.LogRequestAsync(context);
}


//-----------------------------------------------------
// Response |
//-----------------------------------------------------
if (this.logResponse)
{
//-----------------------------------------------------
// Log Response |
//-----------------------------------------------------
await this.LogResponseAsync(context);
}
else
{
//-----------------------------------------------------
// Fire next in chain |
//-----------------------------------------------------
await _next.Invoke(context);
}

}


/// <summary>
/// Log the incoming request
/// </summary>
/// <param name="context"></param>
/// <returns></returns>
private async Task LogRequestAsync(HttpContext context)
{
using (var requestStream = _recyclableMemoryStreamManager.GetStream())
{
var origPosition = context.Request.Body.Position;
context.Request.Body.Position = 0;
await context.Request.Body.CopyToAsync(requestStream);
context.Request.Body.Position = origPosition;

_logger.LogInformation($"Http Request Information:{Environment.NewLine}" +
$"Schema:{context.Request.Scheme} " +
$"Host: {context.Request.Host} " +
$"Path: {context.Request.Path} " +
$"QueryString: {context.Request.QueryString} " +
$"Request Body: {ReadStreamInChunks(requestStream)}");
}
}


private async Task LogResponseAsync(HttpContext context)
{
//--------------------------------------------------------
// Keep reference to original body |
//--------------------------------------------------------
var originalBody = context.Response.Body;

using (var responseStream = _recyclableMemoryStreamManager.GetStream())
{
//-----------------------------------------------------
// Replace existing with read/write stream |
//-----------------------------------------------------
context.Response.Body = responseStream;

//-----------------------------------------------------
// Fire next in chain |
//-----------------------------------------------------
await _next.Invoke(context);

//-----------------------------------------------------
// Get Result Back |
//-----------------------------------------------------
string resp = ReadStreamInChunks(responseStream);

//------------------------------------------------------
// Write it back to original Response Body |
//------------------------------------------------------

byte[] response = Encoding.ASCII.GetBytes(resp);
await originalBody.WriteAsync(response, 0, response.Length);

//------------------------------------------------------
// Log It |
//------------------------------------------------------

_logger.LogInformation($"Http Response Information:{Environment.NewLine}" +
$"Schema:{context.Request.Scheme} " +
$"Host: {context.Request.Host} " +
$"Path: {context.Request.Path} " +
$"QueryString: {context.Request.QueryString} " +
$"Response Body: {resp}");
}

//-----------------------------------------------------------
// Restore |
//-----------------------------------------------------------
context.Response.Body = originalBody;
}


private static string ReadStreamInChunks(Stream stream)
{
stream.Seek(0, SeekOrigin.Begin);
string result;
using (var textWriter = new StringWriter())
using (var reader = new StreamReader(stream))
{
var readChunk = new char[ReadChunkBufferLength];
int readChunkLength;
//do while: is useful for the last iteration in case readChunkLength < chunkLength
do
{
readChunkLength = reader.ReadBlock(readChunk, 0, ReadChunkBufferLength);
textWriter.Write(readChunk, 0, readChunkLength);
} while (readChunkLength > 0);

result = textWriter.ToString();
}

return result;
}

}