Streams: “An Abstraction of a Sequence of Bytes”

According to Microsoft docs, a stream is an abstraction that represents a sequence of bytes. Streams are a fundamental concept in .NET for handling input and output operations. They allow you to read from and write to various data sources in a consistent manner.

We're not going to dive too deeply into streams right now - we're just going to go over the basic use cases.

Common Streams

There are several common types of streams provided by .NET, each serving different purposes:

• FileStream: Used for reading and writing to files on disk.
• MemoryStream: Provides a stream for storing data in memory.
• NetworkStream: Allows reading and writing over network connections.

Basic Operations with Streams

Reading from Files

Streams are commonly used to read from files. You can perform both synchronous and asynchronous read operations.

using (var fileStream = new FileStream("file.txt", FileMode.Open))
{
    using (var reader = new StreamReader(fileStream))
    {
        string content = reader.ReadToEnd();
        Console.WriteLine(content);
    }
}

Writing to Files

using (var fileStream = new FileStream("file.txt", FileMode.Create))
{
    using (var writer = new StreamWriter(fileStream))
    {
        writer.Write("Hello, World!");
    }
} 

Reading stuff to Memory

byte[] data = Encoding.UTF8.GetBytes("Hello, MemoryStream!");
using (var memoryStream = new MemoryStream(data))
{
    using (var reader = new StreamReader(memoryStream))
    {
        string content = reader.ReadToEnd();
        Console.WriteLine(content);
    }
}

A bit more on MemoryStream

MemoryStream is useful for in-memory data storage. It's a type of stream that stores data in memory, which can be useful for scenarios where you need to work with data in memory without using a file.

Example

byte[] data = Encoding.UTF8.GetBytes("Hello, MemoryStream!");
using (var memoryStream = new MemoryStream(data))
{
    using (var reader = new StreamReader(memoryStream))
    {
        string content = reader.ReadToEnd();
        Console.WriteLine(content);
    }

    byte[] allBytes = memoryStream.ToArray();
    Console.WriteLine(allBytes.Length);
}

Notes on MemoryStream

Avoid Overusing MemoryStream. Be cautious when using MemoryStream for large data. Like List<T>, .NET devs in the wild have overused it - it is useful but often used in a way that defeats the purpose of streams.

The number of times I've seen code like this is just Too Darn Much:

public async Task<IActionResult> GetFile(int fileId)
{
    var fileData = await GetFileFromDatabaseAsync(fileId);

    // This loads the entire file into memory, which is wasteful for large files
    using (var memoryStream = new MemoryStream(fileData))
    {
        return File(memoryStream.ToArray(), "application/octet-stream", "file.txt");
    }
}

Oftentimes, the frameworks we use can support using streams directly:

public async Task<IActionResult> GetFile(int fileId)
{
    var fileStream = await GetFileStreamFromDatabaseAsync(fileId);

    return File(fileStream, "application/octet-stream", "file.txt");
}

If you find yourself using MemoryStream frequently, consider whether you need to load the entire byte stream into memory. For large datasets, it may be more efficient to process the data in chunks or use other types of streams.

To avoid cluttering up memory, you can use temporary file APIs to write data to disk temporarily.

string tempFilePath = Path.GetTempFileName();

try
{
    using (var tempFileStream = new FileStream(tempFilePath, FileMode.Create))
    {
        using (var writer = new StreamWriter(tempFileStream))
        {
            writer.Write("Temporary data");
        }
    }

    // Read from the temporary file
    using (var tempFileStream = new FileStream(tempFilePath, FileMode.Open))
    {
        using (var reader = new StreamReader(tempFileStream))
        {
            string content = reader.ReadToEnd();
            Console.WriteLine(content);
        }
    }
}
finally
{
    // Clean up temporary file
    File.Delete(tempFilePath);
}