<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://timofey256.github.io/feed.xml" rel="self" type="application/atom+xml"/><link href="https://timofey256.github.io/" rel="alternate" type="text/html" hreflang="en"/><updated>2025-08-03T12:49:10+00:00</updated><id>https://timofey256.github.io/feed.xml</id><title type="html">Tymofii’s blog</title><subtitle>A simple, whitespace theme for academics. Based on [*folio](https://github.com/bogoli/-folio) design. </subtitle><entry><title type="html">building an in-memory filesystem driver from scratch</title><link href="https://timofey256.github.io/blog/2025/fsys-driver/" rel="alternate" type="text/html" title="building an in-memory filesystem driver from scratch"/><published>2025-08-01T00:00:00+00:00</published><updated>2025-08-01T00:00:00+00:00</updated><id>https://timofey256.github.io/blog/2025/fsys-driver</id><content type="html" xml:base="https://timofey256.github.io/blog/2025/fsys-driver/"><![CDATA[<p>A file system is what allows us to organize files and directories into hierarchical trees. But how is this actually implemented in Linux?</p> <p>Following the Feynman’s famous quote, we are going to build own in-memory file system driver to understand it. This guide focuses on the practical aspects necessary to get such a filesystem up and running. For a deeper dive, check out the references and the “Further Reading” section at the end.</p> <p>You can find the source code in this <a href="https://github.com/timofey256/ram-file-system">repo</a>.</p> <h2 id="how-do-users-interact-with-filesystems">How Do Users Interact with Filesystems?</h2> <p>Your first thought can be: “Through applications, the shell, or tools like <code class="language-plaintext highlighter-rouge">ls</code> and <code class="language-plaintext highlighter-rouge">vim</code>.” That’s true but let’s go one level deeper, and you’ll find system calls.</p> <p>Whenever a userspace program performs an I/O operation: opening a file, reading data, or writing to disk - it issues a system call such as <code class="language-plaintext highlighter-rouge">open</code>, <code class="language-plaintext highlighter-rouge">read</code>, or <code class="language-plaintext highlighter-rouge">write</code>. These syscalls are the entry points into the kernel.</p> <p>But how does the kernel handle them? How does it know <em>where</em> in memory to write the data, <em>how</em> to create or delete files, or <em>which</em> filesystem should respond?</p> <p>We’re not going to cover syscall mechanics in this post (you can find excellent <a href="https://linux-kernel-labs.github.io/refs/heads/master/lectures/syscalls.html">explanations here</a>), but we’ll explore what happens <em>after</em> a syscall hits the kernel — specifically how the Virtual Filesystem (VFS) bridges this gap.</p> <h2 id="the-virtual-filesystem-vfs">The Virtual Filesystem (VFS)</h2> <p>The Virtual Filesystem is a component of the kernel that handles all system calls related to files and file systems. Think of it as a universal adapter which allows multiple filesystems (ext4, tmpfs, NFS, your custom driver) to coexist and plug into the same syscall interface. VFS takes care of most of the complex and error-prone parts, like caching, buffer management, and pathname resolution but delegates the actual storage and retrieval to your specific filesystem driver.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/filesys-driver/vfs-arch-480.webp 480w,/assets/img/filesys-driver/vfs-arch-800.webp 800w,/assets/img/filesys-driver/vfs-arch-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/filesys-driver/vfs-arch.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> </div> </div> <div class="caption"> High-level overview of how VFS works </div> <h2 id="how-does-the-vfs-interface-look">How Does the VFS Interface Look?</h2> <p>Let’s work from first principles. If you were designing a filesystem interface, you’d want to define:</p> <ol> <li>Metadata about the filesystem itself: its name, block size, max filename length, etc.</li> <li>Operations on the filesystem: how to mount it, unmount it, query statistics, etc.</li> </ol> <p>That’s exactly what Linux does using a structure called <code class="language-plaintext highlighter-rouge">file_system_type</code>.</p> <h4 id="file_system_type-registering-a-filesystem"><code class="language-plaintext highlighter-rouge">file_system_type</code>: Registering a Filesystem</h4> <p>This structure represents a specific type of filesystem (e.g. <code class="language-plaintext highlighter-rouge">ext4</code>, <code class="language-plaintext highlighter-rouge">tmpfs</code>, or <code class="language-plaintext highlighter-rouge">myramfs</code>) and provides the logic for mounting and unmounting it:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">file_system_type</span> <span class="p">{</span>
    <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">name</span><span class="p">;</span>
    <span class="k">struct</span> <span class="n">dentry</span> <span class="o">*</span><span class="p">(</span><span class="o">*</span><span class="n">mount</span><span class="p">)(</span><span class="k">struct</span> <span class="n">file_system_type</span> <span class="o">*</span><span class="p">,</span> <span class="kt">int</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="p">);</span>
    <span class="kt">void</span> <span class="p">(</span><span class="o">*</span><span class="n">kill_sb</span><span class="p">)(</span><span class="k">struct</span> <span class="n">super_block</span> <span class="o">*</span><span class="p">);</span>
    <span class="k">struct</span> <span class="n">module</span> <span class="o">*</span><span class="n">owner</span><span class="p">;</span>
    <span class="c1">// ...</span>
<span class="p">};</span>
</code></pre></div></div> <p>When your driver is loaded, you register this structure with the kernel using <code class="language-plaintext highlighter-rouge">register_filesystem</code>.</p> <h4 id="superblock-mounting-a-filesystem">Superblock: Mounting a Filesystem</h4> <p>Once a filesystem is registered, how does it get <em>used</em>? The answer is: via mounting.</p> <p>Every mounted instance of a filesystem is represented by a <code class="language-plaintext highlighter-rouge">super_block</code> structure, which tracks its root directory, all its inodes, and any internal metadata:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>struct super_block {
    struct list_head s_inodes;   // All inodes in this mount
    struct dentry *s_root;       // Root directory entry
    struct file_system_type *s_type; // Back-pointer to FS driver
    unsigned long s_blocksize;
    unsigned long s_magic;
    const struct super_operations *s_op;
    void *s_fs_info;             // FS-specific data
    // ...
};
</code></pre></div></div> <p>The superblock esentially answers: “What does this filesystem look like once mounted?”</p> <h4 id="inode-representing-files-and-directories">Inode: Representing Files and Directories</h4> <p>Next, we need a way to represent individual files or directories. In Linux, they’re both handled using a structure called an inode.</p> <p>An inode holds metadata like size, permissions, timestamps, and pointers to file content. But importantly — it <strong>doesn’t</strong> store the filename.</p> <p>Why not? Because the same inode can have multiple names (hard links), and we don’t want to duplicate the actual file or its metadata. The filename is managed separately, using a <code class="language-plaintext highlighter-rouge">dentry</code>.</p> <p>Linux kernel implementation of inode is <a href="https://github.com/torvalds/linux/blob/master/fs/ext4/ext4.h#L787">here</a>.</p> <h4 id="dentry-directory-entry">Dentry: Directory Entry</h4> <p>A <strong>dentry</strong> (directory entry) maps a filename to its corresponding inode. You can think of it as the glue between filenames and the actual file content.</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/filesys-driver/dentry-mapping-480.webp 480w,/assets/img/filesys-driver/dentry-mapping-800.webp 800w,/assets/img/filesys-driver/dentry-mapping-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/filesys-driver/dentry-mapping.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> </div> </div> <div class="caption"> Illustration of how dentries map to inodes. </div> <p>Multiple dentries can point to the same inode (e.g., via <code class="language-plaintext highlighter-rouge">ln file linkname</code>), enabling hard links without data duplication. You can inspect inode numbers with <code class="language-plaintext highlighter-rouge">ls -i</code>:</p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">touch </span>file
<span class="nv">$ </span><span class="nb">ln </span>file <span class="nb">link</span>
<span class="nv">$ </span><span class="nb">ls</span> <span class="nt">-i</span>
</code></pre></div></div> <p>Example output:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>49020997 file
49020997 link
</code></pre></div></div> <p>Here’s a how dentry looks like in <a href="https://elixir.bootlin.com/linux/v6.16/source/include/linux/dcache.h#L92">Linux kernel source code</a>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>struct dentry {
    //...
    struct inode             *d_inode;     /* associated inode */
    //...
    struct dentry            *d_parent;    /* dentry object of parent */
    struct qstr              d_name;       /* dentry name */
    //...

    struct dentry_operations *d_op;        /* dentry operations table */
    struct super_block       *d_sb;        /* superblock of file */
    void                     *d_fsdata;    /* filesystem-specific data */
    //...
};
</code></pre></div></div> <h4 id="struct-file-open-file-instances"><code class="language-plaintext highlighter-rouge">struct file</code>: Open File Instances</h4> <p>When a file is opened (via <code class="language-plaintext highlighter-rouge">open()</code> syscall), the kernel creates a <code class="language-plaintext highlighter-rouge">struct file</code> instance. It tracks:</p> <ul> <li>The current offset (<code class="language-plaintext highlighter-rouge">f_pos</code>)</li> <li>Flags like read/write mode</li> <li>A pointer to the file’s operations (read, write, seek, etc.)</li> <li>A pointer to the inode and private data</li> </ul> <p>This is what gets passed to your <code class="language-plaintext highlighter-rouge">read</code>, <code class="language-plaintext highlighter-rouge">write</code>, and <code class="language-plaintext highlighter-rouge">ioctl</code> handlers.</p> <h4 id="how-it-all-interacts-together">How it all interacts together?</h4> <p>Here’s how everything connects:</p> <div class="row mt-3"> <div class="col-sm mt-3 mt-md-0"> <figure> <picture> <source class="responsive-img-srcset" srcset="/assets/img/filesys-driver/ds-structure-480.webp 480w,/assets/img/filesys-driver/ds-structure-800.webp 800w,/assets/img/filesys-driver/ds-structure-1400.webp 1400w," type="image/webp" sizes="95vw"/> <img src="/assets/img/filesys-driver/ds-structure.png" class="img-fluid rounded z-depth-1" width="100%" height="auto" loading="eager" onerror="this.onerror=null; $('.responsive-img-srcset').remove();"/> </picture> </figure> </div> </div> <div class="caption"> How different data structures are linked together. </div> <h2 id="implementation-define-file-system-and-its-superblock">Implementation: define file system and its superblock</h2> <p>Now we can start implementing our filesystem driver. We’ll begin from scratch by defining the file system type:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="k">const</span> <span class="k">struct</span> <span class="n">super_operations</span> <span class="n">rf_sops</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">statfs</span>      <span class="o">=</span> <span class="n">simple_statfs</span><span class="p">,</span>  <span class="c1">// default function from lib</span>
    <span class="p">.</span><span class="n">drop_inode</span>  <span class="o">=</span> <span class="n">generic_delete_inode</span><span class="p">,</span> <span class="c1">// default function from lib</span>
    <span class="p">.</span><span class="n">evict_inode</span> <span class="o">=</span> <span class="n">rf_evict</span> <span class="c1">// custom function; see implementation in the source</span>
<span class="p">};</span>

<span class="k">static</span> <span class="kt">int</span> <span class="nf">rf_fill_super</span><span class="p">(</span><span class="k">struct</span> <span class="n">super_block</span> <span class="o">*</span><span class="n">sb</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">,</span> <span class="kt">int</span> <span class="n">silent</span><span class="p">)</span>
<span class="p">{</span>
    <span class="n">sb</span><span class="o">-&gt;</span><span class="n">s_op</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">rf_sops</span><span class="p">;</span>
    <span class="n">sb</span><span class="o">-&gt;</span><span class="n">s_magic</span> <span class="o">=</span> <span class="n">RAMFSC_MAGIC</span><span class="p">;</span>
    <span class="n">sb</span><span class="o">-&gt;</span><span class="n">s_time_gran</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>

    <span class="c1">// initialize root directory</span>
    <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">root</span><span class="p">;</span>
    <span class="n">root</span> <span class="o">=</span> <span class="n">rf_make_inode</span><span class="p">(</span><span class="n">sb</span><span class="p">,</span> <span class="n">S_IFDIR</span> <span class="o">|</span> <span class="mo">0755</span><span class="p">);</span> <span class="c1">// custom function</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">root</span><span class="p">)</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>
    <span class="n">root</span><span class="o">-&gt;</span><span class="n">i_op</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">rf_dir_iops</span><span class="p">;</span>

    <span class="n">sb</span><span class="o">-&gt;</span><span class="n">s_root</span> <span class="o">=</span> <span class="n">d_make_root</span><span class="p">(</span><span class="n">root</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">sb</span><span class="o">-&gt;</span><span class="n">s_root</span><span class="p">)</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>

    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>

<span class="k">static</span> <span class="k">struct</span> <span class="n">dentry</span> <span class="o">*</span><span class="nf">rf_mount</span><span class="p">(</span><span class="k">struct</span> <span class="n">file_system_type</span> <span class="o">*</span><span class="n">t</span><span class="p">,</span>
                               <span class="kt">int</span> <span class="n">flags</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">dev</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">return</span> <span class="n">mount_nodev</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">flags</span><span class="p">,</span> <span class="n">data</span><span class="p">,</span> <span class="n">rf_fill_super</span><span class="p">);</span>
<span class="p">}</span>

<span class="k">static</span> <span class="k">struct</span> <span class="n">file_system_type</span> <span class="n">rf_fs_type</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">owner</span>   <span class="o">=</span> <span class="n">THIS_MODULE</span><span class="p">,</span>
    <span class="p">.</span><span class="n">name</span>    <span class="o">=</span> <span class="s">"myramfs"</span><span class="p">,</span>
    <span class="p">.</span><span class="n">mount</span>   <span class="o">=</span> <span class="n">rf_mount</span><span class="p">,</span>
    <span class="p">.</span><span class="n">kill_sb</span> <span class="o">=</span> <span class="n">kill_litter_super</span><span class="p">,</span>
<span class="p">};</span>

<span class="k">static</span> <span class="kt">int</span> <span class="n">__init</span> <span class="nf">rf_init</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>   <span class="p">{</span> <span class="k">return</span> <span class="n">register_filesystem</span><span class="p">(</span><span class="o">&amp;</span><span class="n">rf_fs_type</span><span class="p">);</span> <span class="p">}</span>
<span class="k">static</span> <span class="kt">void</span> <span class="n">__exit</span> <span class="nf">rf_exit</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>  <span class="p">{</span> <span class="n">unregister_filesystem</span><span class="p">(</span><span class="o">&amp;</span><span class="n">rf_fs_type</span><span class="p">);</span> <span class="p">}</span>
</code></pre></div></div> <p>The VFS provides two functions for registering and unregistering a filesystem: <code class="language-plaintext highlighter-rouge">register_filesystem</code> and <code class="language-plaintext highlighter-rouge">unregister_filesystem</code>. Both accept a <code class="language-plaintext highlighter-rouge">file_system_type</code> structure, which defines the owner, the name of the driver (remember this—we’ll use it when mounting later!), and two function pointers invoked during mount and unmount operations, respectively.</p> <p>Let’s examine those functions more closely. <code class="language-plaintext highlighter-rouge">rf_mount</code> is called during the mounting process. It simply delegates to the standard <code class="language-plaintext highlighter-rouge">mount_nodev</code>, which initializes the superblock and then calls <code class="language-plaintext highlighter-rouge">rf_fill_super</code> to finish the setup.</p> <p><code class="language-plaintext highlighter-rouge">rf_fill_super</code> performs two main tasks: it completes the superblock initialization and attaches the root directory to it.</p> <h2 id="how-to-operate-under-root">How to operate under <code class="language-plaintext highlighter-rouge">root</code>?</h2> <p><code class="language-plaintext highlighter-rouge">root</code> is a directory, so we need to define how to look up files, create new files, and create subdirectories under it. All of this is specified in <code class="language-plaintext highlighter-rouge">rf_dir_iops</code> (remember how we assigned it when creating the root inode?). Let’s take a closer look:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="k">const</span> <span class="k">struct</span> <span class="n">inode_operations</span> <span class="n">rf_dir_iops</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">lookup</span> <span class="o">=</span> <span class="n">simple_lookup</span><span class="p">,</span>
    <span class="p">.</span><span class="n">create</span> <span class="o">=</span> <span class="n">rf_create</span><span class="p">,</span>
    <span class="p">.</span><span class="n">setattr</span> <span class="o">=</span> <span class="n">rf_setattr</span><span class="p">,</span>
    <span class="p">.</span><span class="n">mkdir</span> <span class="o">=</span> <span class="n">rf_mkdir</span><span class="p">,</span>
<span class="p">};</span>
</code></pre></div></div> <p>For now, we define just four operations:</p> <ul> <li><code class="language-plaintext highlighter-rouge">lookup</code>: a default VFS function used to resolve names to dentries.</li> <li><code class="language-plaintext highlighter-rouge">create</code>: used to create regular files.</li> <li><code class="language-plaintext highlighter-rouge">setattr</code>: used internally by the VFS to set inode attributes.</li> <li><code class="language-plaintext highlighter-rouge">mkdir</code>: used to create directories.</li> </ul> <p>Let’s walk through each of these:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">int</span> <span class="nf">rf_create</span><span class="p">(</span><span class="k">struct</span> <span class="n">mnt_idmap</span> <span class="o">*</span><span class="n">idmap</span><span class="p">,</span> <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">dir</span><span class="p">,</span>
                     <span class="k">struct</span> <span class="n">dentry</span> <span class="o">*</span><span class="n">dentry</span><span class="p">,</span> <span class="n">umode_t</span> <span class="n">mode</span><span class="p">,</span> <span class="n">bool</span> <span class="n">excl</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">ino</span> <span class="o">=</span> <span class="n">rf_make_inode</span><span class="p">(</span><span class="n">dir</span><span class="o">-&gt;</span><span class="n">i_sb</span><span class="p">,</span> <span class="n">S_IFREG</span> <span class="o">|</span> <span class="n">mode</span><span class="p">);</span>
    <span class="k">struct</span> <span class="n">rbuf</span>  <span class="o">*</span><span class="n">rb</span><span class="p">;</span>

    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ino</span><span class="p">)</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>

    <span class="n">rb</span> <span class="o">=</span> <span class="n">kzalloc</span><span class="p">(</span><span class="k">sizeof</span><span class="p">(</span><span class="o">*</span><span class="n">rb</span><span class="p">),</span> <span class="n">GFP_KERNEL</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">rb</span> <span class="o">||</span> <span class="n">rf_reserve</span><span class="p">(</span><span class="n">rb</span><span class="p">,</span> <span class="n">PAGE_SIZE</span><span class="p">))</span> <span class="p">{</span>
        <span class="n">iput</span><span class="p">(</span><span class="n">ino</span><span class="p">);</span>
        <span class="n">kfree</span><span class="p">(</span><span class="n">rb</span><span class="p">);</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>
    <span class="p">}</span>
    <span class="n">ino</span><span class="o">-&gt;</span><span class="n">i_private</span> <span class="o">=</span> <span class="n">rb</span><span class="p">;</span>

    <span class="n">d_add</span><span class="p">(</span><span class="n">dentry</span><span class="p">,</span> <span class="n">ino</span><span class="p">);</span>   <span class="c1">// bind dentry to inode</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <p>When creating a new file, the VFS calls <code class="language-plaintext highlighter-rouge">rf_create</code>. The steps are:</p> <ol> <li>Allocate an inode — the core structure holding file metadata.</li> <li>Since the file will store data, allocate a buffer. We use a simple in-memory buffer type, <code class="language-plaintext highlighter-rouge">rbuf</code>:</li> </ol> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* File RAM buffer */</span>
<span class="k">struct</span> <span class="n">rbuf</span> <span class="p">{</span>
    <span class="kt">char</span>  <span class="o">*</span><span class="n">data</span><span class="p">;</span>
    <span class="kt">size_t</span> <span class="n">size</span><span class="p">;</span>      <span class="c1">// bytes used</span>
    <span class="kt">size_t</span> <span class="n">cap</span><span class="p">;</span>       <span class="c1">// bytes allocated</span>
<span class="p">};</span>
</code></pre></div></div> <p>This is an in-memory filesystem, so we don’t care about persistence. The buffer is allocated using <code class="language-plaintext highlighter-rouge">rf_reserve</code>, which is essentially a wrapper around <code class="language-plaintext highlighter-rouge">malloc</code> — see the source for details.</p> <p>Once memory is allocated, we link the inode to the dentry using <code class="language-plaintext highlighter-rouge">d_add</code>. Now, to the new directories.</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">int</span> <span class="nf">rf_mkdir</span><span class="p">(</span><span class="k">struct</span> <span class="n">mnt_idmap</span> <span class="o">*</span><span class="n">idmap</span><span class="p">,</span> <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">dir</span><span class="p">,</span>
                    <span class="k">struct</span> <span class="n">dentry</span> <span class="o">*</span><span class="n">dentry</span><span class="p">,</span> <span class="n">umode_t</span> <span class="n">mode</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">inode</span><span class="p">;</span>

    <span class="n">inode</span> <span class="o">=</span> <span class="n">rf_make_inode</span><span class="p">(</span><span class="n">dir</span><span class="o">-&gt;</span><span class="n">i_sb</span><span class="p">,</span> <span class="n">S_IFDIR</span> <span class="o">|</span> <span class="n">mode</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">inode</span><span class="p">)</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>

    <span class="n">inode_inc_link_count</span><span class="p">(</span><span class="n">dir</span><span class="p">);</span>
    <span class="n">inode_inc_link_count</span><span class="p">(</span><span class="n">inode</span><span class="p">);</span>

    <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_op</span>  <span class="o">=</span> <span class="o">&amp;</span><span class="n">rf_dir_iops</span><span class="p">;</span>
    <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_fop</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">simple_dir_operations</span><span class="p">;</span>

    <span class="n">d_add</span><span class="p">(</span><span class="n">dentry</span><span class="p">,</span> <span class="n">inode</span><span class="p">);</span>

    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <p>This follows the same basic flow as <code class="language-plaintext highlighter-rouge">rf_create</code>, with one key addition: the two calls to <code class="language-plaintext highlighter-rouge">inode_inc_link_count</code>.</p> <p>What’s happening here?</p> <ul> <li><code class="language-plaintext highlighter-rouge">inode_inc_link_count(inode)</code> handles the <code class="language-plaintext highlighter-rouge">"."</code> link: every directory contains a reference to itself.</li> <li><code class="language-plaintext highlighter-rouge">inode_inc_link_count(dir)</code> accounts for the <code class="language-plaintext highlighter-rouge">".."</code> link: the new directory will reference its parent, and the parent now contains one more subdirectory.</li> </ul> <p>This mirrors how UNIX filesystems track directory link counts — each subdirectory increases its parent’s link count by 1.</p> <p>I’m skipping <code class="language-plaintext highlighter-rouge">rf_setattr</code> here for simplicity. You can check out the implementation in the source.</p> <h2 id="but-how-did-we-allocate-inode">But how did we allocate inode?</h2> <p>When we were creating new files and directories, you may have noticed that the actual allocation of the inode happened somewhere else. In <code class="language-plaintext highlighter-rouge">rf_create</code> and <code class="language-plaintext highlighter-rouge">rf_mkdir</code>, we simply called <code class="language-plaintext highlighter-rouge">rf_make_inode</code>, then added custom metadata or attached buffers. So how was the inode actually allocated?</p> <p>The answer: <code class="language-plaintext highlighter-rouge">rf_make_inode</code> is just a thin wrapper around <code class="language-plaintext highlighter-rouge">new_inode</code>.</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="nf">rf_make_inode</span><span class="p">(</span><span class="k">struct</span> <span class="n">super_block</span> <span class="o">*</span><span class="n">sb</span><span class="p">,</span> <span class="n">umode_t</span> <span class="n">mode</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">inode</span> <span class="o">=</span> <span class="n">new_inode</span><span class="p">(</span><span class="n">sb</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">inode</span><span class="p">)</span>
        <span class="k">return</span> <span class="nb">NULL</span><span class="p">;</span>

    <span class="n">inode_init_owner</span><span class="p">(</span><span class="o">&amp;</span><span class="n">nop_mnt_idmap</span><span class="p">,</span> <span class="n">inode</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">,</span> <span class="n">mode</span><span class="p">);</span>

    <span class="k">if</span> <span class="p">(</span><span class="n">S_ISDIR</span><span class="p">(</span><span class="n">mode</span><span class="p">))</span> <span class="p">{</span>
        <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_op</span>  <span class="o">=</span> <span class="o">&amp;</span><span class="n">simple_dir_inode_operations</span><span class="p">;</span>
        <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_fop</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">simple_dir_operations</span><span class="p">;</span>
    <span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
        <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_fop</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">rf_fops</span><span class="p">;</span>
        <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_mapping</span><span class="o">-&gt;</span><span class="n">a_ops</span> <span class="o">=</span> <span class="o">&amp;</span><span class="n">empty_aops</span><span class="p">;</span>
    <span class="p">}</span>
    <span class="k">return</span> <span class="n">inode</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <p>Based on the mode, we check if this inode represents a directory. If it’s a directory, we assign it default directory operations via <code class="language-plaintext highlighter-rouge">simple_dir_inode_operations</code> and <code class="language-plaintext highlighter-rouge">simple_dir_operations</code>. If it’s a regular file, we assign it our own <code class="language-plaintext highlighter-rouge">rf_fops</code> for file operations and configure the address space operations (<code class="language-plaintext highlighter-rouge">a_ops</code>) using <code class="language-plaintext highlighter-rouge">empty_aops</code>. This disables any page-level caching or backing store because we’re working purely in memory.</p> <h2 id="finally-file-manipulations">Finally, File Manipulations!</h2> <p>Naturally, we want to be able to read from and write to the inodes we’ve created. Let’s define the appropriate file operations.</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="k">const</span> <span class="k">struct</span> <span class="n">file_operations</span> <span class="n">rf_fops</span> <span class="o">=</span> <span class="p">{</span>
    <span class="p">.</span><span class="n">open</span>    <span class="o">=</span> <span class="n">rf_open</span><span class="p">,</span>
    <span class="p">.</span><span class="n">read</span>    <span class="o">=</span> <span class="n">rf_read</span><span class="p">,</span>
    <span class="p">.</span><span class="n">write</span>   <span class="o">=</span> <span class="n">rf_write</span><span class="p">,</span>
    <span class="p">.</span><span class="n">llseek</span>  <span class="o">=</span> <span class="n">generic_file_llseek</span><span class="p">,</span>
    <span class="p">.</span><span class="n">fsync</span>   <span class="o">=</span> <span class="n">rf_fsync</span><span class="p">,</span>
<span class="p">};</span>
</code></pre></div></div> <h4 id="rf_open"><code class="language-plaintext highlighter-rouge">rf_open</code></h4> <p>When a file is opened, we simply attach its associated buffer (stored in the inode) to the file’s private data:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">int</span> <span class="nf">rf_open</span><span class="p">(</span><span class="k">struct</span> <span class="n">inode</span> <span class="o">*</span><span class="n">inode</span><span class="p">,</span> <span class="k">struct</span> <span class="n">file</span> <span class="o">*</span><span class="n">filp</span><span class="p">)</span>
<span class="p">{</span>
    <span class="n">filp</span><span class="o">-&gt;</span><span class="n">private_data</span> <span class="o">=</span> <span class="n">inode</span><span class="o">-&gt;</span><span class="n">i_private</span><span class="p">;</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <h4 id="rf_read"><code class="language-plaintext highlighter-rouge">rf_read</code></h4> <p>To read from a file, we copy data from our in-memory buffer to user space. The buffer is retrieved from <code class="language-plaintext highlighter-rouge">filp-&gt;private_data</code>, which we set in <code class="language-plaintext highlighter-rouge">rf_open</code>:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">ssize_t</span> <span class="nf">rf_read</span><span class="p">(</span><span class="k">struct</span> <span class="n">file</span> <span class="o">*</span><span class="n">f</span><span class="p">,</span> <span class="kt">char</span> <span class="n">__user</span> <span class="o">*</span><span class="n">buf</span><span class="p">,</span>
                       <span class="kt">size_t</span> <span class="n">len</span><span class="p">,</span> <span class="n">loff_t</span> <span class="o">*</span><span class="n">ppos</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">struct</span> <span class="n">rbuf</span> <span class="o">*</span><span class="n">rb</span> <span class="o">=</span> <span class="n">f</span><span class="o">-&gt;</span><span class="n">private_data</span><span class="p">;</span>
    <span class="k">return</span> <span class="n">simple_read_from_buffer</span><span class="p">(</span><span class="n">buf</span><span class="p">,</span> <span class="n">len</span><span class="p">,</span> <span class="n">ppos</span><span class="p">,</span> <span class="n">rb</span><span class="o">-&gt;</span><span class="n">data</span><span class="p">,</span> <span class="n">rb</span><span class="o">-&gt;</span><span class="n">size</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div> <p>This delegates to a kernel helper that handles offset tracking and boundary checking.</p> <h4 id="rf_write"><code class="language-plaintext highlighter-rouge">rf_write</code></h4> <p>Writing is slightly more involved, but still straightforward. We:</p> <ol> <li>Retrieve our buffer from <code class="language-plaintext highlighter-rouge">private_data</code>.</li> <li>Check whether the file is opened in append mode.</li> <li>Calculate the new end offset.</li> <li>Reserve enough space in the buffer.</li> <li>Copy data from user space.</li> <li>Update the offset, buffer size, and inode size.</li> </ol> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">ssize_t</span> <span class="nf">rf_write</span><span class="p">(</span><span class="k">struct</span> <span class="n">file</span> <span class="o">*</span><span class="n">f</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="n">__user</span> <span class="o">*</span><span class="n">buf</span><span class="p">,</span>
                        <span class="kt">size_t</span> <span class="n">len</span><span class="p">,</span> <span class="n">loff_t</span> <span class="o">*</span><span class="n">ppos</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">struct</span> <span class="n">rbuf</span> <span class="o">*</span><span class="n">rb</span> <span class="o">=</span> <span class="n">f</span><span class="o">-&gt;</span><span class="n">private_data</span><span class="p">;</span>

    <span class="k">if</span> <span class="p">(</span><span class="n">f</span><span class="o">-&gt;</span><span class="n">f_flags</span> <span class="o">&amp;</span> <span class="n">O_APPEND</span><span class="p">)</span>
        <span class="o">*</span><span class="n">ppos</span> <span class="o">=</span> <span class="n">rb</span><span class="o">-&gt;</span><span class="n">size</span><span class="p">;</span>

    <span class="n">loff_t</span> <span class="n">end</span> <span class="o">=</span> <span class="o">*</span><span class="n">ppos</span> <span class="o">+</span> <span class="n">len</span><span class="p">;</span>

    <span class="k">if</span> <span class="p">(</span><span class="n">end</span> <span class="o">&gt;</span> <span class="n">INT_MAX</span><span class="p">)</span> <span class="c1">// sanity check</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">EFBIG</span><span class="p">;</span>
    <span class="k">if</span> <span class="p">(</span><span class="n">rf_reserve</span><span class="p">(</span><span class="n">rb</span><span class="p">,</span> <span class="n">end</span><span class="p">))</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">ENOMEM</span><span class="p">;</span>
    <span class="k">if</span> <span class="p">(</span><span class="n">copy_from_user</span><span class="p">(</span><span class="n">rb</span><span class="o">-&gt;</span><span class="n">data</span> <span class="o">+</span> <span class="o">*</span><span class="n">ppos</span><span class="p">,</span> <span class="n">buf</span><span class="p">,</span> <span class="n">len</span><span class="p">))</span>
        <span class="k">return</span> <span class="o">-</span><span class="n">EFAULT</span><span class="p">;</span>

    <span class="o">*</span><span class="n">ppos</span> <span class="o">+=</span> <span class="n">len</span><span class="p">;</span>
    <span class="n">rb</span><span class="o">-&gt;</span><span class="n">size</span> <span class="o">=</span> <span class="n">max_t</span><span class="p">(</span><span class="kt">size_t</span><span class="p">,</span> <span class="n">rb</span><span class="o">-&gt;</span><span class="n">size</span><span class="p">,</span> <span class="n">end</span><span class="p">);</span>
    <span class="n">i_size_write</span><span class="p">(</span><span class="n">file_inode</span><span class="p">(</span><span class="n">f</span><span class="p">),</span> <span class="n">rb</span><span class="o">-&gt;</span><span class="n">size</span><span class="p">);</span> <span class="c1">// updates inode's size</span>
    <span class="k">return</span> <span class="n">len</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <h4 id="fsync"><code class="language-plaintext highlighter-rouge">fsync</code></h4> <p>If you open a file in <code class="language-plaintext highlighter-rouge">vim</code> and try to save it, the editor will call the <code class="language-plaintext highlighter-rouge">fsync</code> syscall to flush file contents to disk. If <code class="language-plaintext highlighter-rouge">fsync</code> is unimplemented, this operation would fail. Since we’re building an in-memory filesystem, there’s nothing to flush. But we still need to handle the call:</p> <div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="kt">int</span> <span class="nf">rf_fsync</span><span class="p">(</span><span class="k">struct</span> <span class="n">file</span> <span class="o">*</span><span class="n">file</span><span class="p">,</span> <span class="n">loff_t</span> <span class="n">start</span><span class="p">,</span> <span class="n">loff_t</span> <span class="n">end</span><span class="p">,</span> <span class="kt">int</span> <span class="n">datasync</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div> <h2 id="further-reading">Further reading</h2> <ul> <li><a href="https://linux-kernel-labs.github.io/refs/heads/master/lectures/fs.html">Linux kernel labs: Filesystem Management</a> - excellent notes describing Linux Filesystem Management.</li> <li><a href="https://linux-kernel-labs.github.io/refs/heads/master/labs/filesystems_part1.html">Linux kernel labs: Filesystem drivers (Part 1)</a> - check out their labs on drivers too!</li> <li><a href="https://lwn.net/Articles/57369/">Creating Linux virtual filesystems</a> - older but very simple guide on basic filesys driver.</li> <li><a href="https://sysprog21.github.io/lkmpg/">Longer guide on writing Linux Kernel modules</a>.</li> <li><a href="https://aeb.win.tue.nl/linux/lk/lk-8.html">The Linux Kernel</a> by Andries Brouwer.</li> <li><a href="https://nano-chicken.blogspot.com/2020/05/linux-kernel181-my-first-filesystem.html">Some chinese blogpost where I took diagrams from</a>.</li> </ul>]]></content><author><name></name></author><category term="programming"/><category term="linux,"/><category term="kernel,"/><category term="c"/><summary type="html"><![CDATA[let's understand how linux filesystem works and how to write your own]]></summary></entry><entry><title type="html">a cheatsheet of C# multithreading</title><link href="https://timofey256.github.io/blog/2025/multithreading-in-csharp/" rel="alternate" type="text/html" title="a cheatsheet of C# multithreading"/><published>2025-07-15T00:00:00+00:00</published><updated>2025-07-15T00:00:00+00:00</updated><id>https://timofey256.github.io/blog/2025/multithreading-in-csharp</id><content type="html" xml:base="https://timofey256.github.io/blog/2025/multithreading-in-csharp/"><![CDATA[<p>I’ve recently taken some time to refresh my understanding of multithreading in C#. This post is a collection of notes I wrote down along the way. It’s not a tutorial - more of a compact reference I (and you) can return to when there’s a need to recall how things work.</p> <p>It covers core concepts like threads, tasks, synchronization primitives, data parallelism, async streams, and more. I’ve included practical examples and benchmarks where they make sense, but the goal isn’t to teach everything from scratch—just to gather the important patterns, behaviors, and gotchas in one place.</p> <hr/> <h2 id="threads">Threads</h2> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">MethodName</span><span class="p">()</span> <span class="p">{</span> <span class="cm">/* do smth */</span> <span class="p">}</span>

<span class="n">Thread</span> <span class="n">t1</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Thread</span><span class="p">(</span><span class="n">MethodName</span><span class="p">);</span>
<span class="n">t1</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
<span class="n">t1</span><span class="p">.</span><span class="nf">Join</span><span class="p">();</span>
<span class="n">t1</span><span class="p">.</span><span class="nf">Interrupt</span><span class="p">();</span>                 <span class="c1">// abort if blocked</span>
<span class="n">t1</span><span class="p">.</span><span class="n">IsBackground</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="n">t1</span><span class="p">.</span><span class="n">Priority</span> <span class="p">=</span> <span class="n">ThreadPriority</span><span class="p">.</span><span class="n">Highest</span><span class="p">;</span>

<span class="kt">int</span> <span class="n">id</span> <span class="p">=</span> <span class="n">Thread</span><span class="p">.</span><span class="n">CurrentThread</span><span class="p">.</span><span class="n">ManagedThreadId</span><span class="p">;</span>
</code></pre></div></div> <p><em>Default stack size of a .NET thread is 1 MiB</em>. You can change it by passing <code class="language-plaintext highlighter-rouge">stackSize</code> in the ctor:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">t</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Thread</span><span class="p">(</span><span class="n">SomeMethod</span><span class="p">,</span> <span class="n">stackSize</span><span class="p">:</span> <span class="m">512</span> <span class="p">*</span> <span class="m">1024</span><span class="p">);</span>
</code></pre></div></div> <hr/> <h2 id="tasks-await-and-concurrency">Tasks, Await, and Concurrency</h2> <p>Tasks represent units of work, while Threads are C# abstractions over OS Threads.</p> <h4 id="threadsvstasks">Threads vs Tasks</h4> <table> <thead> <tr> <th> </th> <th><code class="language-plaintext highlighter-rouge">Thread</code></th> <th><code class="language-plaintext highlighter-rouge">Task</code></th> </tr> </thead> <tbody> <tr> <td>Represents</td> <td>running OS thread</td> <td>unit of work</td> </tr> <tr> <td>Create with</td> <td><code class="language-plaintext highlighter-rouge">new Thread</code></td> <td><code class="language-plaintext highlighter-rouge">Task.Run</code> (ThreadPool)</td> </tr> <tr> <td>Use for</td> <td>long‑lived, STA, custom stack</td> <td>bursty CPU or async I/O</td> </tr> </tbody> </table> <h4 id="configureawait">ConfigureAwait</h4> <p>How await works by default</p> <ul> <li>The compiler inserts an awaiter for every await task;.</li> <li>Just before suspension it captures the current <ul> <li>SynchronizationContext (if one exists, e.g., WinForms/WPF UI thread, classic ASP.NET request thread) or</li> <li>the current TaskScheduler (usually the ThreadPool).</li> </ul> </li> <li>When task completes, the continuation is posted back to that captured context so your method keeps executing on the original thread.</li> </ul> <p><code class="language-plaintext highlighter-rouge">ConfigureAwait(bool)</code> lets you to set if you want this capture to happen or not. So we can roughly make the following distinction:</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UI apps (WinForms/WPF)    : ConfigureAwait(true)
Libraries / ASP.NET Core  : ConfigureAwait(false)
</code></pre></div></div> <h4 id="task-continuations">Task continuations</h4> <p>A continuation is code that runs after a <code class="language-plaintext highlighter-rouge">Task</code> completes, without blocking the current thread. You define it using the <code class="language-plaintext highlighter-rouge">ContinueWith</code> method:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Task</span> <span class="n">first</span> <span class="p">=</span> <span class="nf">SomeTask</span><span class="p">();</span>
<span class="n">first</span><span class="p">.</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">t</span> <span class="p">=&gt;</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"This runs after SomeTask completes."</span><span class="p">));</span>
</code></pre></div></div> <p>This creates a dependency chain between tasks.</p> <h4 id="be-careful-with-chained-task-continuations">Be careful with chained task continuations</h4> <p>Can you spot a problem here? What the below code will print? Is it expected?</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="n">Task</span> <span class="nf">DoFirstThing</span><span class="p">()</span>
<span class="p">{</span>
    <span class="k">return</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"First thing done"</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">static</span> <span class="n">Task</span> <span class="nf">DoSecondThing</span><span class="p">()</span>
<span class="p">{</span>
    <span class="k">return</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Delay</span><span class="p">(</span><span class="m">1000</span><span class="p">).</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">_</span> <span class="p">=&gt;</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"Second thing done"</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">static</span> <span class="n">Task</span> <span class="nf">DoThirdThing</span><span class="p">()</span>
<span class="p">{</span>
    <span class="k">return</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"Third thing done"</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span>
<span class="p">{</span>
    <span class="nf">DoFirstThing</span><span class="p">().</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">_</span> <span class="p">=&gt;</span> <span class="nf">DoSecondThing</span><span class="p">()).</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">_</span> <span class="p">=&gt;</span> <span class="nf">DoThirdThing</span><span class="p">());</span>

    <span class="n">Console</span><span class="p">.</span><span class="nf">ReadLine</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div> <details><summary>Open see the answer</summary> <p>Program output:</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>First thing done.
Third thing done.
Second thing done.
</code></pre></div></div> <p>Order is wrong because of the incorrect usage of continuations. In <code class="language-plaintext highlighter-rouge">DoSecondThing()</code> we return a <code class="language-plaintext highlighter-rouge">Task&lt;Task&gt;</code>, because <code class="language-plaintext highlighter-rouge">ContinueWith</code> always returns <code class="language-plaintext highlighter-rouge">Task&lt;T&gt;</code> where <code class="language-plaintext highlighter-rouge">T</code> is the return type of the delegate which is Task in this case. The next <code class="language-plaintext highlighter-rouge">ContinueWith</code> will just unwrap the first <code class="language-plaintext highlighter-rouge">Task</code>, getting <code class="language-plaintext highlighter-rouge">Task</code>, so will not await anything.</p> <p>The correct usage would be:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">DoFirstThing</span><span class="p">().</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">_</span> <span class="p">=&gt;</span> <span class="nf">DoSecondThing</span><span class="p">()).</span><span class="nf">Unwrap</span><span class="p">().</span><span class="nf">ContinueWith</span><span class="p">(</span><span class="n">_</span> <span class="p">=&gt;</span> <span class="nf">DoThirdThing</span><span class="p">());</span>
</code></pre></div></div> <p>Alternatively, you could use 3 <code class="language-plaintext highlighter-rouge">await</code> lines, or nested continuations.</p> </details> <h4 id="timeout--cancellation">Timeout / cancellation</h4> <p>You can add timeouts to awaits:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">await</span> <span class="nf">DoWorkAsync</span><span class="p">().</span><span class="nf">WaitAsync</span><span class="p">(</span><span class="n">TimeSpan</span><span class="p">.</span><span class="nf">FromSeconds</span><span class="p">(</span><span class="m">2</span><span class="p">));</span>
</code></pre></div></div> <h4 id="taskrunvstaskfactorystartnew"><code class="language-plaintext highlighter-rouge">Task.Run</code> vs <code class="language-plaintext highlighter-rouge">Task.Factory.StartNew</code></h4> <p><code class="language-plaintext highlighter-rouge">Task.Run(f)</code> ≈ <code class="language-plaintext highlighter-rouge">Task.Factory.StartNew(f, TaskCreationOptions.DenyChildAttach).Unwrap()</code>.</p> <hr/> <h2 id="data-parallelism">Data Parallelism</h2> <p>Data parallelism refers to scenarios in which the same operation is performed concurrently (that is, in parallel) on elements in a source collection or array. In data parallel operations, the source collection is partitioned so that multiple threads can operate on different segments concurrently.</p> <p>Let’s try to implement some data parallelism using raw tasks - we will have an array <code class="language-plaintext highlighter-rouge">bitmaps</code> and we want to apply <code class="language-plaintext highlighter-rouge">ApplyFilter</code> on each of its elements.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span> <span class="n">degree</span> <span class="p">=</span> <span class="n">Environment</span><span class="p">.</span><span class="n">ProcessorCount</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">chunk</span>  <span class="p">=</span> <span class="p">(</span><span class="kt">int</span><span class="p">)</span><span class="n">Math</span><span class="p">.</span><span class="nf">Ceiling</span><span class="p">(</span><span class="n">bitmaps</span><span class="p">.</span><span class="n">Length</span> <span class="p">/</span> <span class="p">(</span><span class="kt">double</span><span class="p">)</span><span class="n">degree</span><span class="p">);</span>

<span class="kt">var</span> <span class="n">tasks</span> <span class="p">=</span> <span class="n">Enumerable</span><span class="p">.</span><span class="nf">Range</span><span class="p">(</span><span class="m">0</span><span class="p">,</span> <span class="n">degree</span><span class="p">).</span><span class="nf">Select</span><span class="p">(</span><span class="n">core</span> <span class="p">=&gt;</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="kt">int</span> <span class="n">start</span> <span class="p">=</span> <span class="n">core</span> <span class="p">*</span> <span class="n">chunk</span><span class="p">;</span>
    <span class="kt">int</span> <span class="n">end</span>   <span class="p">=</span> <span class="n">Math</span><span class="p">.</span><span class="nf">Min</span><span class="p">(</span><span class="n">start</span> <span class="p">+</span> <span class="n">chunk</span><span class="p">,</span> <span class="n">bitmaps</span><span class="p">.</span><span class="n">Length</span><span class="p">);</span>

    <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="n">start</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">end</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
        <span class="n">bitmaps</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="p">=</span> <span class="nf">ApplyFilter</span><span class="p">(</span><span class="n">bitmaps</span><span class="p">[</span><span class="n">i</span><span class="p">]);</span>
<span class="p">})).</span><span class="nf">ToArray</span><span class="p">();</span>

<span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="n">tasks</span><span class="p">);</span>
</code></pre></div></div> <p>As you see, we had to deal with a lot of low-level things - specifying chunk sizes, manually calculating indices, etc. It’s easy to make bugs when writing code like this.</p> <p>Fortunately, .NET Task Parallel Library (TPL) has <code class="language-plaintext highlighter-rouge">System.Threading.Tasks.Parallel</code> class. Using it we can rewrite the above code piece much more concisely:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Parallel</span><span class="p">.</span><span class="nf">For</span><span class="p">(</span><span class="m">0</span><span class="p">,</span> <span class="n">bitmaps</span><span class="p">.</span><span class="n">Length</span><span class="p">,</span> <span class="n">i</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="n">bitmaps</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="p">=</span> <span class="nf">ApplyFilter</span><span class="p">(</span><span class="n">bitmaps</span><span class="p">[</span><span class="n">i</span><span class="p">]);</span>
<span class="p">});</span>
</code></pre></div></div> <h4 id="parallel-also-supports-async"><code class="language-plaintext highlighter-rouge">Parallel</code> also supports Async</h4> <p>Fetching a resource w/o <code class="language-plaintext highlighter-rouge">Parallel</code>:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">using</span> <span class="nn">var</span> <span class="n">sem</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SemaphoreSlim</span><span class="p">(</span><span class="m">2</span><span class="p">);</span>
<span class="kt">var</span> <span class="n">tasks</span> <span class="p">=</span> <span class="n">urls</span><span class="p">.</span><span class="nf">Select</span><span class="p">(</span><span class="k">async</span> <span class="n">url</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="k">await</span> <span class="n">sem</span><span class="p">.</span><span class="nf">WaitAsync</span><span class="p">();</span>
    <span class="k">try</span>   <span class="p">{</span> <span class="k">await</span> <span class="nf">FetchAsync</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">CancellationToken</span><span class="p">.</span><span class="n">None</span><span class="p">);</span> <span class="p">}</span>
    <span class="k">finally</span> <span class="p">{</span> <span class="n">sem</span><span class="p">.</span><span class="nf">Release</span><span class="p">();</span> <span class="p">}</span>
<span class="p">});</span>
<span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="n">tasks</span><span class="p">);</span>
</code></pre></div></div> <p>With <code class="language-plaintext highlighter-rouge">Parallel</code>:</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">await</span> <span class="n">Parallel</span><span class="p">.</span><span class="nf">ForEachAsync</span><span class="p">(</span><span class="n">urls</span><span class="p">,</span>
    <span class="k">new</span> <span class="n">ParallelOptions</span> <span class="p">{</span> <span class="n">MaxDegreeOfParallelism</span> <span class="p">=</span> <span class="m">2</span> <span class="p">},</span>
    <span class="k">async</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">ct</span><span class="p">)</span> <span class="p">=&gt;</span> <span class="k">await</span> <span class="nf">FetchAsync</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">ct</span><span class="p">));</span>
</code></pre></div></div> <h4 id="parallelforeachasync-vs-taskwhenall"><code class="language-plaintext highlighter-rouge">Parallel.ForEachAsync</code> vs <code class="language-plaintext highlighter-rouge">Task.WhenAll</code></h4> <p>You can see above a useful feature of <code class="language-plaintext highlighter-rouge">Parallel.ForEachAsync</code>: it let’s you specify degree of your parallelism.</p> <hr/> <h2 id="atomicity-visibility-and-locks">Atomicity, Visibility, and Locks</h2> <h4 id="volatile">volatile</h4> <p>Guarantees reads/writes hit main memory and adds fences that stop CPU/compiler re‑ordering.</p> <h4 id="systemthreadinginterlocked"><code class="language-plaintext highlighter-rouge">System.Threading.Interlocked</code></h4> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span> <span class="n">counter</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
<span class="n">Interlocked</span><span class="p">.</span><span class="nf">Increment</span>  <span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">);</span>
<span class="n">Interlocked</span><span class="p">.</span><span class="nf">Decrement</span>  <span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">);</span>
<span class="n">Interlocked</span><span class="p">.</span><span class="nf">Add</span>        <span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">,</span> <span class="m">5</span><span class="p">);</span>
<span class="kt">int</span> <span class="n">old</span> <span class="p">=</span> <span class="n">Interlocked</span><span class="p">.</span><span class="nf">Exchange</span><span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">,</span> <span class="m">100</span><span class="p">);</span>
<span class="n">Interlocked</span><span class="p">.</span><span class="nf">CompareExchange</span><span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">,</span> <span class="m">10</span><span class="p">,</span> <span class="m">5</span><span class="p">);</span> <span class="c1">// set to 10 only when counter == 5</span>
</code></pre></div></div> <h4 id="interlockedvslockbenchmark">Interlocked vs lock (benchmark)</h4> <p>Why should you use <code class="language-plaintext highlighter-rouge">Interlocked</code> instead of <code class="language-plaintext highlighter-rouge">lock</code> whenever you can? Let’s compare their performance.</p> <details><summary>Open to see the benchmark code</summary> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CompareInterlockedAndLockProgram</span>
<span class="p">{</span>
    <span class="k">static</span> <span class="kt">int</span> <span class="n">N</span> <span class="p">=</span> <span class="m">1_000_000</span><span class="p">;</span> <span class="c1">// Increments per thread</span>
    <span class="k">static</span> <span class="kt">int</span> <span class="n">T</span> <span class="p">=</span> <span class="m">4</span><span class="p">;</span>         <span class="c1">// Number of threads</span>

    <span class="k">static</span> <span class="k">void</span> <span class="nf">RunWithInterlocked</span><span class="p">()</span>
    <span class="p">{</span>
        <span class="kt">int</span> <span class="n">counter</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>

        <span class="k">void</span> <span class="nf">Work</span><span class="p">()</span>
        <span class="p">{</span>
            <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
                <span class="n">Interlocked</span><span class="p">.</span><span class="nf">Increment</span><span class="p">(</span><span class="k">ref</span> <span class="n">counter</span><span class="p">);</span>
        <span class="p">}</span>

        <span class="n">Thread</span><span class="p">[]</span> <span class="n">threads</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Thread</span><span class="p">[</span><span class="n">T</span><span class="p">];</span>
        <span class="n">Stopwatch</span> <span class="n">sw</span> <span class="p">=</span> <span class="n">Stopwatch</span><span class="p">.</span><span class="nf">StartNew</span><span class="p">();</span>

        <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">T</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span> <span class="n">threads</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Thread</span><span class="p">(</span><span class="n">Work</span><span class="p">);</span>
        <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">t</span> <span class="k">in</span> <span class="n">threads</span><span class="p">)</span> <span class="n">t</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
        <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">t</span> <span class="k">in</span> <span class="n">threads</span><span class="p">)</span> <span class="n">t</span><span class="p">.</span><span class="nf">Join</span><span class="p">();</span>

        <span class="n">sw</span><span class="p">.</span><span class="nf">Stop</span><span class="p">();</span>
        <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Interlocked: </span><span class="p">{</span><span class="n">sw</span><span class="p">.</span><span class="n">ElapsedMilliseconds</span><span class="p">}</span><span class="s"> ms | Final = </span><span class="p">{</span><span class="n">counter</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
    <span class="p">}</span>

    <span class="k">static</span> <span class="k">void</span> <span class="nf">RunWithLock</span><span class="p">()</span>
    <span class="p">{</span>
        <span class="kt">int</span> <span class="n">counter</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span>
        <span class="kt">object</span> <span class="n">lockObj</span> <span class="p">=</span> <span class="k">new</span> <span class="kt">object</span><span class="p">();</span>

        <span class="k">void</span> <span class="nf">Work</span><span class="p">()</span>
        <span class="p">{</span>
            <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
            <span class="p">{</span>
                <span class="k">lock</span> <span class="p">(</span><span class="n">lockObj</span><span class="p">)</span>
                <span class="p">{</span>
                    <span class="n">counter</span><span class="p">++;</span>
                <span class="p">}</span>
            <span class="p">}</span>
        <span class="p">}</span>

        <span class="n">Thread</span><span class="p">[]</span> <span class="n">threads</span> <span class="p">=</span> <span class="k">new</span> <span class="n">Thread</span><span class="p">[</span><span class="n">T</span><span class="p">];</span>
        <span class="n">Stopwatch</span> <span class="n">sw</span> <span class="p">=</span> <span class="n">Stopwatch</span><span class="p">.</span><span class="nf">StartNew</span><span class="p">();</span>

        <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">T</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span> <span class="n">threads</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Thread</span><span class="p">(</span><span class="n">Work</span><span class="p">);</span>
        <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">t</span> <span class="k">in</span> <span class="n">threads</span><span class="p">)</span> <span class="n">t</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
        <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">t</span> <span class="k">in</span> <span class="n">threads</span><span class="p">)</span> <span class="n">t</span><span class="p">.</span><span class="nf">Join</span><span class="p">();</span>

        <span class="n">sw</span><span class="p">.</span><span class="nf">Stop</span><span class="p">();</span>
        <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Lock:       </span><span class="p">{</span><span class="n">sw</span><span class="p">.</span><span class="n">ElapsedMilliseconds</span><span class="p">}</span><span class="s"> ms | Final = </span><span class="p">{</span><span class="n">counter</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
    <span class="p">}</span>

    <span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">Start</span><span class="p">()</span>
    <span class="p">{</span>
        <span class="nf">RunWithInterlocked</span><span class="p">();</span>
        <span class="nf">RunWithLock</span><span class="p">();</span>
    <span class="p">}</span>
<span class="p">}</span>

<span class="k">class</span> <span class="nc">Program</span> <span class="p">{</span>
    <span class="k">public</span> <span class="k">static</span> <span class="k">void</span> <span class="nf">Main</span><span class="p">(</span><span class="kt">string</span><span class="p">[]</span> <span class="n">args</span><span class="p">)</span> <span class="p">{</span>
       <span class="n">CompareInterlockedAndLockProgram</span><span class="p">.</span><span class="nf">Start</span><span class="p">();</span>
    <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div> </details> <p>Typical run on 4 threads × 1 M increments each:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Interlocked: 42 ms | Final = 4000000
Lock:       101 ms | Final = 4000000
</code></pre></div></div> <hr/> <h2 id="basic-synchronization-primitives">Basic Synchronization Primitives</h2> <h3 id="semaphoreslim">SemaphoreSlim</h3> <p>Control how many concurrent callers enter a critical section. <code class="language-plaintext highlighter-rouge">SemaphoreSlim</code> is also async‑friendly. Limit a CPU‑bound method to 2 parallel executions.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">SemaphoreSlim</span> <span class="n">sem</span> <span class="p">=</span> <span class="k">new</span><span class="p">(</span><span class="m">2</span><span class="p">,</span> <span class="m">2</span><span class="p">);</span>

<span class="k">async</span> <span class="n">Task</span> <span class="nf">WorkAsync</span><span class="p">()</span>
<span class="p">{</span>
    <span class="k">await</span> <span class="n">sem</span><span class="p">.</span><span class="nf">WaitAsync</span><span class="p">();</span>           <span class="c1">// ↓ counter or wait</span>
    <span class="k">try</span>   <span class="p">{</span> <span class="k">await</span> <span class="nf">DoHeavyStuff</span><span class="p">();</span> <span class="p">}</span>
    <span class="k">finally</span> <span class="p">{</span> <span class="n">sem</span><span class="p">.</span><span class="nf">Release</span><span class="p">();</span> <span class="p">}</span>       <span class="c1">// ↑ counter</span>
<span class="p">}</span>
</code></pre></div></div> <hr/> <h4 id="manualreseteventslim">ManualResetEventSlim</h4> <p>One thread signals many waiters; stays signalled until reset. Let all tasks start only after a warm‑up step.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">go</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">ManualResetEventSlim</span><span class="p">(</span><span class="k">false</span><span class="p">);</span>

<span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="p">{</span> <span class="nf">WarmUp</span><span class="p">();</span> <span class="n">go</span><span class="p">.</span><span class="nf">Set</span><span class="p">();</span> <span class="p">});</span>        <span class="c1">// signal ON</span>

<span class="n">Parallel</span><span class="p">.</span><span class="nf">For</span><span class="p">(</span><span class="m">0</span><span class="p">,</span> <span class="m">8</span><span class="p">,</span> <span class="n">i</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="n">go</span><span class="p">.</span><span class="nf">Wait</span><span class="p">();</span>                                  <span class="c1">// all block here</span>
    <span class="nf">Process</span><span class="p">(</span><span class="n">i</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div> <hr/> <h4 id="autoreseteventslim">AutoResetEventSlim</h4> <p>Signal wakes <strong>one</strong> waiter, then auto‑resets. Classic producer–consumer hand‑shake.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">evt</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">AutoResetEventSlim</span><span class="p">(</span><span class="k">false</span><span class="p">);</span>

<span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span>                        <span class="c1">// producer</span>
<span class="p">{</span>
    <span class="k">while</span> <span class="p">(</span><span class="nf">Produce</span><span class="p">(</span><span class="k">out</span> <span class="kt">var</span> <span class="n">item</span><span class="p">))</span>
    <span class="p">{</span>
        <span class="n">queue</span><span class="p">.</span><span class="nf">Enqueue</span><span class="p">(</span><span class="n">item</span><span class="p">);</span>
        <span class="n">evt</span><span class="p">.</span><span class="nf">Set</span><span class="p">();</span>                    <span class="c1">// wake one consumer</span>
    <span class="p">}</span>
<span class="p">});</span>

<span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span>                        <span class="c1">// consumer</span>
<span class="p">{</span>
    <span class="k">while</span> <span class="p">(</span><span class="k">true</span><span class="p">)</span>
    <span class="p">{</span>
        <span class="n">evt</span><span class="p">.</span><span class="nf">Wait</span><span class="p">();</span>                   <span class="c1">// waits again after each Set()</span>
        <span class="k">if</span> <span class="p">(</span><span class="n">queue</span><span class="p">.</span><span class="nf">TryDequeue</span><span class="p">(</span><span class="k">out</span> <span class="kt">var</span> <span class="n">x</span><span class="p">))</span> <span class="nf">Use</span><span class="p">(</span><span class="n">x</span><span class="p">);</span>
    <span class="p">}</span>
<span class="p">});</span>
</code></pre></div></div> <hr/> <h4 id="countdownevent">CountdownEvent</h4> <p><strong>Fork / join</strong>: continue when <em>n</em> signals have arrived. Fire off N raw threads, wait for all to finish.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span> <span class="n">N</span> <span class="p">=</span> <span class="m">3</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">cd</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">CountdownEvent</span><span class="p">(</span><span class="n">N</span><span class="p">);</span>

<span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
    <span class="k">new</span> <span class="nf">Thread</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="p">{</span> <span class="nf">Work</span><span class="p">();</span> <span class="n">cd</span><span class="p">.</span><span class="nf">Signal</span><span class="p">();</span> <span class="p">}).</span><span class="nf">Start</span><span class="p">();</span>

<span class="n">cd</span><span class="p">.</span><span class="nf">Wait</span><span class="p">();</span>                            <span class="c1">// resumes when Signal() called N times</span>
</code></pre></div></div> <hr/> <h4 id="barrier">Barrier</h4> <p>Multi‑phase workflow: every participant must reach the barrier before any enters the next phase. 4 workers perform Stage 1 → Stage 2 → Stage 3 in lock‑step.</p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">barrier</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">Barrier</span><span class="p">(</span><span class="m">4</span><span class="p">,</span> <span class="n">_</span> <span class="p">=&gt;</span> <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">"Phase done"</span><span class="p">));</span>

<span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="m">4</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
    <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span>
    <span class="p">{</span>
        <span class="nf">Stage1</span><span class="p">();</span> <span class="n">barrier</span><span class="p">.</span><span class="nf">SignalAndWait</span><span class="p">();</span>
        <span class="nf">Stage2</span><span class="p">();</span> <span class="n">barrier</span><span class="p">.</span><span class="nf">SignalAndWait</span><span class="p">();</span>
        <span class="nf">Stage3</span><span class="p">();</span> <span class="n">barrier</span><span class="p">.</span><span class="nf">SignalAndWait</span><span class="p">();</span>
    <span class="p">});</span>
</code></pre></div></div> <hr/> <h2 id="producer-consumer-pattern">Producer-Consumer Pattern</h2> <p>Producer-Consumer decouples data creation from data processing: producers push items into a shared buffer while consumers pull them out. In .NET you can choose a blocking queue (<code class="language-plaintext highlighter-rouge">BlockingCollection&lt;T&gt;</code>) or the modern async‑friendly <code class="language-plaintext highlighter-rouge">Channel&lt;T&gt;</code> (you should use the latter most of the time).</p> <h4 id="blocking-collection-threadblocking">Blocking Collection (thread‑blocking)</h4> <aside><p> We use <code>Thread.Sleep</code> here for demonstration. In real code, consider using <code>await Task.Delay</code> for non-blocking waiting. </p></aside> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">queue</span> <span class="p">=</span> <span class="k">new</span> <span class="n">BlockingCollection</span><span class="p">&lt;</span><span class="kt">int</span><span class="p">&gt;(</span><span class="n">boundedCapacity</span><span class="p">:</span> <span class="m">5</span><span class="p">);</span>

<span class="kt">var</span> <span class="n">producer</span> <span class="p">=</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="m">10</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span> <span class="p">{</span>
<span class="n">queue</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span><span class="n">i</span><span class="p">);</span> <span class="c1">// Blocks if full</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Produced: </span><span class="p">{</span><span class="n">i</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="n">Thread</span><span class="p">.</span><span class="nf">Sleep</span><span class="p">(</span><span class="m">100</span><span class="p">);</span>  
 <span class="p">}</span>
<span class="n">queue</span><span class="p">.</span><span class="nf">CompleteAdding</span><span class="p">();</span> <span class="c1">// Signal no more items</span>
<span class="p">});</span>

<span class="c1">// Consumer</span>
<span class="kt">var</span> <span class="n">consumer</span> <span class="p">=</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(()</span> <span class="p">=&gt;</span> <span class="p">{</span>
<span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">item</span> <span class="k">in</span> <span class="n">queue</span><span class="p">.</span><span class="nf">GetConsumingEnumerable</span><span class="p">())</span> <span class="p">{</span>
<span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"\tConsumed: </span><span class="p">{</span><span class="n">item</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
<span class="n">Thread</span><span class="p">.</span><span class="nf">Sleep</span><span class="p">(</span><span class="m">150</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">});</span>

</code></pre></div></div> <h4 id="channelt-asyncawait">Channel&lt;T&gt; (async/await)</h4> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">channel</span> <span class="p">=</span> <span class="n">Channel</span><span class="p">.</span><span class="n">CreateUnbounded</span><span class="p">&lt;</span><span class="kt">int</span><span class="p">&gt;();</span>

<span class="kt">var</span> <span class="n">producer</span> <span class="p">=</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(</span><span class="k">async</span> <span class="p">()</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="p">=</span> <span class="m">0</span><span class="p">;</span> <span class="n">i</span> <span class="p">&lt;</span> <span class="m">10</span><span class="p">;</span> <span class="n">i</span><span class="p">++)</span>
    <span class="p">{</span>
        <span class="k">await</span> <span class="n">channel</span><span class="p">.</span><span class="n">Writer</span><span class="p">.</span><span class="nf">WriteAsync</span><span class="p">(</span><span class="n">i</span><span class="p">);</span>
        <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"Produced: </span><span class="p">{</span><span class="n">i</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
        <span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Delay</span><span class="p">(</span><span class="m">100</span><span class="p">);</span>
    <span class="p">}</span>
    <span class="n">channel</span><span class="p">.</span><span class="n">Writer</span><span class="p">.</span><span class="nf">Complete</span><span class="p">();</span> <span class="c1">// signal end</span>
<span class="p">});</span>

<span class="kt">var</span> <span class="n">consumer</span> <span class="p">=</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Run</span><span class="p">(</span><span class="k">async</span> <span class="p">()</span> <span class="p">=&gt;</span>
<span class="p">{</span>
    <span class="k">await</span> <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">item</span> <span class="k">in</span> <span class="n">channel</span><span class="p">.</span><span class="n">Reader</span><span class="p">.</span><span class="nf">ReadAllAsync</span><span class="p">())</span>
    <span class="p">{</span>
        <span class="n">Console</span><span class="p">.</span><span class="nf">WriteLine</span><span class="p">(</span><span class="s">$"\tConsumed: </span><span class="p">{</span><span class="n">item</span><span class="p">}</span><span class="s">"</span><span class="p">);</span>
        <span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">Delay</span><span class="p">(</span><span class="m">150</span><span class="p">);</span>
    <span class="p">}</span>
<span class="p">});</span>

<span class="k">await</span> <span class="n">Task</span><span class="p">.</span><span class="nf">WhenAll</span><span class="p">(</span><span class="n">producer</span><span class="p">,</span> <span class="n">consumer</span><span class="p">);</span>
</code></pre></div></div> <p>Choose bounded channels for <code class="language-plaintext highlighter-rouge">Wait</code>, <code class="language-plaintext highlighter-rouge">DropOldest</code>, etc. Always call <code class="language-plaintext highlighter-rouge">CompleteAdding()</code> / <code class="language-plaintext highlighter-rouge">Writer.Complete()</code>.</p> <hr/> <h2 id="streaming-datausing-iasyncenumerable">Streaming Data using IAsyncEnumerable</h2> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">async</span> <span class="n">IAsyncEnumerable</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">&gt;</span> <span class="nf">ReadLargeFileAsync</span><span class="p">(</span><span class="kt">string</span> <span class="n">path</span><span class="p">)</span>
<span class="p">{</span>
    <span class="k">using</span> <span class="nn">var</span> <span class="n">r</span> <span class="p">=</span> <span class="n">File</span><span class="p">.</span><span class="nf">OpenText</span><span class="p">(</span><span class="n">path</span><span class="p">);</span>
    <span class="k">while</span> <span class="p">(!</span><span class="n">r</span><span class="p">.</span><span class="n">EndOfStream</span><span class="p">)</span>
        <span class="k">yield</span> <span class="k">return</span> <span class="k">await</span> <span class="n">r</span><span class="p">.</span><span class="nf">ReadLineAsync</span><span class="p">();</span>
<span class="p">}</span>

<span class="k">await</span> <span class="k">foreach</span> <span class="p">(</span><span class="kt">var</span> <span class="n">line</span> <span class="k">in</span> <span class="nf">ReadLargeFileAsync</span><span class="p">(</span><span class="s">"log.txt"</span><span class="p">))</span>
    <span class="nf">Process</span><span class="p">(</span><span class="n">line</span><span class="p">);</span>
</code></pre></div></div> <hr/> <h2 id="async-compatible-initialization">Async-Compatible Initialization</h2> <p><em>Interface pattern</em></p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">interface</span> <span class="nc">IAsyncInitializable</span> <span class="p">{</span> <span class="n">Task</span> <span class="nf">InitializeAsync</span><span class="p">();</span> <span class="p">}</span>
</code></pre></div></div> <p><em>Factory pattern</em></p> <div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">svc</span> <span class="p">=</span> <span class="k">await</span> <span class="n">MyService</span><span class="p">.</span><span class="nf">CreateAsync</span><span class="p">();</span>
</code></pre></div></div>]]></content><author><name></name></author><category term="programming"/><category term="csharp"/><summary type="html"><![CDATA[notes i've taken while refreshing my knowledge]]></summary></entry></feed>