diff --git a/about/index.html b/about/index.html
index 8de594f..810c342 100644
--- a/about/index.html
+++ b/about/index.html
@@ -6,11 +6,11 @@
     Link to heading
   
 
-I&rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++ Since then more than 15 years has passed, and I managed to get an Master degree in computer science and statistics from the University of Economics in Bratislava. I had to opportunity to do build machine learning and distributed systems for early to mid sage startups. Currently my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Data, Code and Breaking Stuff"><meta name=twitter:description content=" About Link to heading I’m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++ Since then more than 15 years has passed, and I managed to get an Master degree in computer science and statistics from the University of Economics in Bratislava. I had to opportunity to do build machine learning and distributed systems for early to mid sage startups. Currently my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta property="og:url" content="https://n1o.github.io/about/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Data, Code and Breaking Stuff"><meta property="og:description" content=" About Link to heading I’m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++ Since then more than 15 years has passed, and I managed to get an Master degree in computer science and statistics from the University of Economics in Bratislava. I had to opportunity to do build machine learning and distributed systems for early to mid sage startups. Currently my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta property="og:locale" content="en"><meta property="og:type" content="article"><link rel=canonical href=https://n1o.github.io/about/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
+I&rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++. Since then nearly 20 years have passed, and I managed to get a Master degree in Computer Science and Statistics from the University of Economics in Bratislava. I had the opportunity to do machine learning and build distributed systems for early to mid stage startups. Right now my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Data, Code and Breaking Stuff"><meta name=twitter:description content=" About Link to heading I’m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++. Since then nearly 20 years have passed, and I managed to get a Master degree in Computer Science and Statistics from the University of Economics in Bratislava. I had the opportunity to do machine learning and build distributed systems for early to mid stage startups. Right now my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta property="og:url" content="https://n1o.github.io/about/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Data, Code and Breaking Stuff"><meta property="og:description" content=" About Link to heading I’m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned Bloodshed C++. Since then nearly 20 years have passed, and I managed to get a Master degree in Computer Science and Statistics from the University of Economics in Bratislava. I had the opportunity to do machine learning and build distributed systems for early to mid stage startups. Right now my day job is at Paiqo tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation."><meta property="og:locale" content="en"><meta property="og:type" content="article"><link rel=canonical href=https://n1o.github.io/about/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
 </a><input type=checkbox id=menu-toggle>
 <label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/about/>About</a></li><li class=navigation-item><a class=navigation-link href=/posts/>Blog</a></li><li class=navigation-item><a class=navigation-link href=/awesome-t5/>Awesome T5</a></li><li class=navigation-item><a class=navigation-link href=/awesome-ssm/>Awesome SSM</a></li><li class=navigation-item><a class=navigation-link href=/projects/>Projects</a></li><li class=navigation-item><a class=navigation-link href=/contact/>Contact me</a></li></ul></section></nav><div class=content><section class="container page"><article><header><h1 class=title><a class=title-link href=https://n1o.github.io/about/></a></h1></header><h1 id=about>About
 <a class=heading-link href=#about><i class="fa-solid fa-link" aria-hidden=true title="Link to heading"></i>
-<span class=sr-only>Link to heading</span></a></h1><p>I&rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned <a href=https://www.bloodshed.net/ class=external-link target=_blank rel=noopener>Bloodshed C++</a> Since then more than 15 years has passed, and I managed to get an Master degree in computer science and statistics from the University of Economics in Bratislava. I had to opportunity to do build machine learning and distributed systems for early to mid sage startups. Currently my day job is at <a href=https://paiqo.com/ class=external-link target=_blank rel=noopener>Paiqo</a> tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation.</p><p>My professional interests are:</p><ul><li>Machine Learning</li><li>High performance and distributed computing</li><li>Reverse Engineering and Vulnerability Exploitation</li><li>Secure Coding</li></ul><p>As of my personal life, I am married (Still fiding it hard to figure out why she said yes!). I am also an father of three sons, cat person and a book worm committed to life long geek-hood.</p></article></section><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.css integrity=sha384-vKruj+a13U8yHIkAyGgK1J3ArTLzrFGBbBc0tDp4ad/EyewESeXE/Iv67Aj8gKZ0 crossorigin=anonymous><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.js integrity=sha384-PwRUT/YqbnEjkZO0zZxNqcxACrXe+j766U2amXcgMg5457rve2Y7I6ZJSm2A0mS4 crossorigin=anonymous></script><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/contrib/auto-render.min.js integrity=sha384-+VBxd3r6XgURycqtZ117nYw44OOcIax56Z4dCRWbxyPt0Koah1uHoK0o4+/RRE05 crossorigin=anonymous onload='renderMathInElement(document.body,{delimiters:[{left:"$$",right:"$$",display:!0},{left:"$",right:"$",display:!1},{left:"\\(",right:"\\)",display:!1},{left:"\\[",right:"\\]",display:!0}]})'></script></div><footer class=footer><section class=container>©
+<span class=sr-only>Link to heading</span></a></h1><p>I&rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned <a href=https://www.bloodshed.net/ class=external-link target=_blank rel=noopener>Bloodshed C++</a>. Since then nearly 20 years have passed, and I managed to get a Master degree in Computer Science and Statistics from the University of Economics in Bratislava. I had the opportunity to do machine learning and build distributed systems for early to mid stage startups. Right now my day job is at <a href=https://paiqo.com/ class=external-link target=_blank rel=noopener>Paiqo</a> tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation.</p><p>My professional interests are:</p><ul><li>Machine Learning</li><li>High performance and distributed computing</li><li>Reverse Engineering and Vulnerability Exploitation</li><li>Secure Coding</li></ul><p>As for my personal life, I am married (Still finding it hard to figure out why she said yes!). I am also a father of three sons, cat person and a book worm committed to life long geek-hood.</p></article></section><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.css integrity=sha384-vKruj+a13U8yHIkAyGgK1J3ArTLzrFGBbBc0tDp4ad/EyewESeXE/Iv67Aj8gKZ0 crossorigin=anonymous><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/katex.min.js integrity=sha384-PwRUT/YqbnEjkZO0zZxNqcxACrXe+j766U2amXcgMg5457rve2Y7I6ZJSm2A0mS4 crossorigin=anonymous></script><script defer src=https://cdn.jsdelivr.net/npm/katex@0.16.4/dist/contrib/auto-render.min.js integrity=sha384-+VBxd3r6XgURycqtZ117nYw44OOcIax56Z4dCRWbxyPt0Koah1uHoK0o4+/RRE05 crossorigin=anonymous onload='renderMathInElement(document.body,{delimiters:[{left:"$$",right:"$$",display:!0},{left:"$",right:"$",display:!1},{left:"\\(",right:"\\)",display:!1},{left:"\\[",right:"\\]",display:!0}]})'></script></div><footer class=footer><section class=container>©
 2020 -
 2024
 n1o_c0rTx
diff --git a/index.html b/index.html
index c41ddb1..543ba70 100644
--- a/index.html
+++ b/index.html
@@ -1,5 +1,5 @@
 <!doctype html><html lang=en><head><title>Data, Code and Breaking Stuff</title>
-<meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=color-scheme content="light dark"><meta name=author content="n1o_c0rTx"><meta name=description content="Data, Code and Breaking Stuff"><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Data, Code and Breaking Stuff"><meta name=twitter:description content="Data, Code and Breaking Stuff"><meta property="og:url" content="https://n1o.github.io/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Data, Code and Breaking Stuff"><meta property="og:description" content="Data, Code and Breaking Stuff"><meta property="og:locale" content="en"><meta property="og:type" content="website"><link rel=canonical href=https://n1o.github.io/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5><link rel=alternate type=application/rss+xml href=/index.xml title="Data, Code and Breaking Stuff"><meta name=generator content="Hugo 0.134.1"></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
+<meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=color-scheme content="light dark"><meta name=author content="n1o_c0rTx"><meta name=description content="Data, Code and Breaking Stuff"><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Data, Code and Breaking Stuff"><meta name=twitter:description content="Data, Code and Breaking Stuff"><meta property="og:url" content="https://n1o.github.io/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Data, Code and Breaking Stuff"><meta property="og:description" content="Data, Code and Breaking Stuff"><meta property="og:locale" content="en"><meta property="og:type" content="website"><link rel=canonical href=https://n1o.github.io/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5><link rel=alternate type=application/rss+xml href=/index.xml title="Data, Code and Breaking Stuff"><meta name=generator content="Hugo 0.134.2"></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
 </a><input type=checkbox id=menu-toggle>
 <label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/about/>About</a></li><li class=navigation-item><a class=navigation-link href=/posts/>Blog</a></li><li class=navigation-item><a class=navigation-link href=/awesome-t5/>Awesome T5</a></li><li class=navigation-item><a class=navigation-link href=/awesome-ssm/>Awesome SSM</a></li><li class=navigation-item><a class=navigation-link href=/projects/>Projects</a></li><li class=navigation-item><a class=navigation-link href=/contact/>Contact me</a></li></ul></section></nav><div class=content><section class="container centered"><div class=about><div class=avatar><img src="https://www.gravatar.com/avatar/4f3b911bf8c48473de66f919a4d219e2?s=240&d=mp" alt=gravatar></div><h1>n1o_c0rTx</h1><h2>Data, Code and Breaking Stuff</h2><ul><li><a href=https://github.com/n1o aria-label=Github><i class="fa fa-github" aria-hidden=true></i></a></li><li><a href=https://twitter.com/n1o_cr0Tx aria-label=Twitter><i class="fa fa-twitter" aria-hidden=true></i></a></li><li><a href=https://www.linkedin.com/in/marek-bar%C3%A1k-31977a55/ aria-label=LinkedIn><i class="fa fa-linkedin" aria-hidden=true></i></a></li><li><a href=https://medium.com/@n1o_cr0Tx aria-label=Medium><i class="fa fa-medium" aria-hidden=true></i></a></li><li><a href=https://n1o.github.com aria-label=RSS rel=alternate type=application/rss+xml><i class="fa fa-rss" aria-hidden=true></i></a></li></ul></div></section></div><footer class=footer><section class=container>©
 2020 -
diff --git a/index.xml b/index.xml
index f53c333..de35d55 100644
--- a/index.xml
+++ b/index.xml
@@ -108,7 +108,8 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create &lt;a href="https://codebreakers.re" class="external-link" target="_blank" rel="noopener">CodeBreakers&lt;/a>
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p>
+&lt;p>I found two notable research papers where &lt;a href="https://n1o.github.io/posts/t5-the-old-new-thing/" >T5&lt;/a> has been applied to reverse engineering are &lt;a href="https://arxiv.org/abs/2301.01701" class="external-link" target="_blank" rel="noopener">BinT5&lt;/a> and &lt;a href="https://www.semanticscholar.org/paper/HexT5%3A-Unified-Pre-Training-for-Stripped-Binary-Xiong-Chen/04c3fccfe01f42afe18dcdb027385f350ab3c9d1" class="external-link" target="_blank" rel="noopener">HexT5&lt;/a>. Before we dive deep into the details of these papers, let&amp;rsquo;s first explore the basics of reverse engineering.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
  Abstract
  &lt;a class="heading-link" href="#abstract">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
@@ -189,7 +190,15 @@ However, it does have a limitation - its context length is restricted to 512 tok
  &lt;span class="sr-only">Link to heading&lt;/span>
  &lt;/a>
 &lt;/h1>
-&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p></description></item><item><title>Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations</title><link>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</link><pubDate>Mon, 16 Jan 2023 17:37:36 +0100</pubDate><guid>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</guid><description>&lt;h1 id="the-problem-it-solves">
+&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p>
+&lt;h1 id="language-modeling-requirements">
+ Language modeling requirements
+ &lt;a class="heading-link" href="#language-modeling-requirements">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;p>The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic &lt;a href="ttps://d2l.ai/chapter_attention-mechanisms-and-transformers/transformer.html" >Transformer&lt;/a>, it has proven to be extremely effective. The reason for its success is explored in the paper &amp;ldquo;&lt;a href="https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html" class="external-link" target="_blank" rel="noopener">In-Context Learning and Induction Heads&lt;/a>.&amp;rdquo; The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:&lt;/p></description></item><item><title>Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations</title><link>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</link><pubDate>Mon, 16 Jan 2023 17:37:36 +0100</pubDate><guid>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</guid><description>&lt;h1 id="the-problem-it-solves">
  The problem it solves
  &lt;a class="heading-link" href="#the-problem-it-solves">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
@@ -197,7 +206,22 @@ However, it does have a limitation - its context length is restricted to 512 tok
  &lt;/a>
 &lt;/h1>
 &lt;p>Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:&lt;/p>
-&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p></description></item><item><title>Hierarchical Probabilistic Matrix Factorization</title><link>https://n1o.github.io/posts/pooled_matrix_factorization/</link><pubDate>Thu, 17 Dec 2020 10:19:42 +0100</pubDate><guid>https://n1o.github.io/posts/pooled_matrix_factorization/</guid><description>&lt;p>Probabilistic Matrix factorization is a simple but useful model for matrix imputation. The main idea is to decompose a tall and wide matrix into a product of two matrices, one tall and thin and one short and wide.&lt;/p>
+&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p>
+&lt;h1 id="an-example-and-some-details">
+ An example and some details
+ &lt;a class="heading-link" href="#an-example-and-some-details">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;h2 id="example">
+ Example
+ &lt;a class="heading-link" href="#example">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h2>
+&lt;p>As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient&amp;rsquo;s CRP levels again.&lt;/p></description></item><item><title>Hierarchical Probabilistic Matrix Factorization</title><link>https://n1o.github.io/posts/pooled_matrix_factorization/</link><pubDate>Thu, 17 Dec 2020 10:19:42 +0100</pubDate><guid>https://n1o.github.io/posts/pooled_matrix_factorization/</guid><description>&lt;p>Probabilistic Matrix factorization is a simple but useful model for matrix imputation. The main idea is to decompose a tall and wide matrix into a product of two matrices, one tall and thin and one short and wide.&lt;/p>
 &lt;p>$$
 R_{n\times m} = U_{m \times d} \cdot V_{d \times n}
 $$&lt;/p>
@@ -218,7 +242,7 @@ $$&lt;/p></description></item><item><title/><link>https://n1o.github.io/about/</
  &lt;span class="sr-only">Link to heading&lt;/span>
  &lt;/a>
 &lt;/h1>
-&lt;p>I&amp;rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned &lt;a href="https://www.bloodshed.net/" class="external-link" target="_blank" rel="noopener">Bloodshed C++&lt;/a> Since then more than 15 years has passed, and I managed to get an Master degree in computer science and statistics from the University of Economics in Bratislava. I had to opportunity to do build machine learning and distributed systems for early to mid sage startups. Currently my day job is at &lt;a href="https://paiqo.com/" class="external-link" target="_blank" rel="noopener">Paiqo&lt;/a> tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation.&lt;/p></description></item><item><title/><link>https://n1o.github.io/contact/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://n1o.github.io/contact/</guid><description>&lt;h1 id="contact">
+&lt;p>I&amp;rsquo;m Marek Barak, I started my journey into computer science when I was 15 by coding in C++, without having access to the internet and in good old fashioned &lt;a href="https://www.bloodshed.net/" class="external-link" target="_blank" rel="noopener">Bloodshed C++&lt;/a>. Since then nearly 20 years have passed, and I managed to get a Master degree in Computer Science and Statistics from the University of Economics in Bratislava. I had the opportunity to do machine learning and build distributed systems for early to mid stage startups. Right now my day job is at &lt;a href="https://paiqo.com/" class="external-link" target="_blank" rel="noopener">Paiqo&lt;/a> tackling anything Data related. At nights I am an independent Cyber Security Researcher figuring out how to apply Machine Learning to Cyber Security especially in the field of Reverse Engineering and Vulnerability Exploitation.&lt;/p></description></item><item><title/><link>https://n1o.github.io/contact/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://n1o.github.io/contact/</guid><description>&lt;h1 id="contact">
  Contact
  &lt;a class="heading-link" href="#contact">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
diff --git a/posts/continuous-time-modeling-of-counterfatual-outcomes/index.html b/posts/continuous-time-modeling-of-counterfatual-outcomes/index.html
index a2ebb06..02a16bd 100644
--- a/posts/continuous-time-modeling-of-counterfatual-outcomes/index.html
+++ b/posts/continuous-time-modeling-of-counterfatual-outcomes/index.html
@@ -7,9 +7,26 @@
   
 
 Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:
-If I perform a hypothetical sequence of interventions how will my time series evolve?"><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations"><meta name=twitter:description content=" The problem it solves Link to heading Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:
-If I perform a hypothetical sequence of interventions how will my time series evolve?"><meta property="og:url" content="https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations"><meta property="og:description" content=" The problem it solves Link to heading Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:
-If I perform a hypothetical sequence of interventions how will my time series evolve?"><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2023-01-16T17:37:36+01:00"><meta property="article:modified_time" content="2023-01-16T17:37:36+01:00"><meta property="article:tag" content="Causality"><meta property="article:tag" content="Differential_equation"><link rel=canonical href=https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
+If I perform a hypothetical sequence of interventions how will my time series evolve?
+
+  An example and some details
+  
+    
+    Link to heading
+  
+
+
+  Example
+  
+    
+    Link to heading
+  
+
+As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient&rsquo;s CRP levels again."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations"><meta name=twitter:description content=" The problem it solves Link to heading Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:
+If I perform a hypothetical sequence of interventions how will my time series evolve?
+An example and some details Link to heading Example Link to heading As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient’s CRP levels again."><meta property="og:url" content="https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations"><meta property="og:description" content=" The problem it solves Link to heading Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:
+If I perform a hypothetical sequence of interventions how will my time series evolve?
+An example and some details Link to heading Example Link to heading As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient’s CRP levels again."><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2023-01-16T17:37:36+01:00"><meta property="article:modified_time" content="2023-01-16T17:37:36+01:00"><meta property="article:tag" content="Causality"><meta property="article:tag" content="Differential_equation"><link rel=canonical href=https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
 </a><input type=checkbox id=menu-toggle>
 <label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/about/>About</a></li><li class=navigation-item><a class=navigation-link href=/posts/>Blog</a></li><li class=navigation-item><a class=navigation-link href=/awesome-t5/>Awesome T5</a></li><li class=navigation-item><a class=navigation-link href=/awesome-ssm/>Awesome SSM</a></li><li class=navigation-item><a class=navigation-link href=/projects/>Projects</a></li><li class=navigation-item><a class=navigation-link href=/contact/>Contact me</a></li></ul></section></nav><div class=content><section class="container post"><article><header><div class=post-title><h1 class=title><a class=title-link href=https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/>Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations</a></h1></div><div class=post-meta><div class=date><span class=posted-on><i class="fa-solid fa-calendar" aria-hidden=true></i>
 <time datetime=2023-01-16T17:37:36+01:00>January 16, 2023
diff --git a/posts/hungry-hungry-hippos/index.html b/posts/hungry-hungry-hippos/index.html
index 2169146..9818bca 100644
--- a/posts/hungry-hungry-hippos/index.html
+++ b/posts/hungry-hungry-hippos/index.html
@@ -6,7 +6,17 @@
     Link to heading
   
 
-By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Paper overview: Hungry Hungry Hippos: Towards Language Modeling with State Space Models"><meta name=twitter:description content=" High level overview Link to heading By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware."><meta property="og:url" content="https://n1o.github.io/posts/hungry-hungry-hippos/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Paper overview: Hungry Hungry Hippos: Towards Language Modeling with State Space Models"><meta property="og:description" content=" High level overview Link to heading By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware."><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2023-02-06T09:39:03+01:00"><meta property="article:modified_time" content="2023-02-06T09:39:03+01:00"><meta property="article:tag" content="NLP"><meta property="article:tag" content="SSM"><link rel=canonical href=https://n1o.github.io/posts/hungry-hungry-hippos/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
+By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.
+
+  Language modeling requirements
+  
+    
+    Link to heading
+  
+
+The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic Transformer, it has proven to be extremely effective. The reason for its success is explored in the paper &ldquo;In-Context Learning and Induction Heads.&rdquo; The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:"><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="Paper overview: Hungry Hungry Hippos: Towards Language Modeling with State Space Models"><meta name=twitter:description content=" High level overview Link to heading By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.
+Language modeling requirements Link to heading The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic Transformer, it has proven to be extremely effective. The reason for its success is explored in the paper “In-Context Learning and Induction Heads.” The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:"><meta property="og:url" content="https://n1o.github.io/posts/hungry-hungry-hippos/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="Paper overview: Hungry Hungry Hippos: Towards Language Modeling with State Space Models"><meta property="og:description" content=" High level overview Link to heading By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.
+Language modeling requirements Link to heading The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic Transformer, it has proven to be extremely effective. The reason for its success is explored in the paper “In-Context Learning and Induction Heads.” The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:"><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2023-02-06T09:39:03+01:00"><meta property="article:modified_time" content="2023-02-06T09:39:03+01:00"><meta property="article:tag" content="NLP"><meta property="article:tag" content="SSM"><link rel=canonical href=https://n1o.github.io/posts/hungry-hungry-hippos/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
 </a><input type=checkbox id=menu-toggle>
 <label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/about/>About</a></li><li class=navigation-item><a class=navigation-link href=/posts/>Blog</a></li><li class=navigation-item><a class=navigation-link href=/awesome-t5/>Awesome T5</a></li><li class=navigation-item><a class=navigation-link href=/awesome-ssm/>Awesome SSM</a></li><li class=navigation-item><a class=navigation-link href=/projects/>Projects</a></li><li class=navigation-item><a class=navigation-link href=/contact/>Contact me</a></li></ul></section></nav><div class=content><section class="container post"><article><header><div class=post-title><h1 class=title><a class=title-link href=https://n1o.github.io/posts/hungry-hungry-hippos/>Paper overview: Hungry Hungry Hippos: Towards Language Modeling with State Space Models</a></h1></div><div class=post-meta><div class=date><span class=posted-on><i class="fa-solid fa-calendar" aria-hidden=true></i>
 <time datetime=2023-02-06T09:39:03+01:00>February 6, 2023
diff --git a/posts/index.xml b/posts/index.xml
index 9dfa69a..d70c85d 100644
--- a/posts/index.xml
+++ b/posts/index.xml
@@ -27,7 +27,8 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create &lt;a href="https://codebreakers.re" class="external-link" target="_blank" rel="noopener">CodeBreakers&lt;/a>
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p>
+&lt;p>I found two notable research papers where &lt;a href="https://n1o.github.io/posts/t5-the-old-new-thing/" >T5&lt;/a> has been applied to reverse engineering are &lt;a href="https://arxiv.org/abs/2301.01701" class="external-link" target="_blank" rel="noopener">BinT5&lt;/a> and &lt;a href="https://www.semanticscholar.org/paper/HexT5%3A-Unified-Pre-Training-for-Stripped-Binary-Xiong-Chen/04c3fccfe01f42afe18dcdb027385f350ab3c9d1" class="external-link" target="_blank" rel="noopener">HexT5&lt;/a>. Before we dive deep into the details of these papers, let&amp;rsquo;s first explore the basics of reverse engineering.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
  Abstract
  &lt;a class="heading-link" href="#abstract">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
@@ -60,7 +61,15 @@ However, it does have a limitation - its context length is restricted to 512 tok
  &lt;span class="sr-only">Link to heading&lt;/span>
  &lt;/a>
 &lt;/h1>
-&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p></description></item><item><title>Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations</title><link>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</link><pubDate>Mon, 16 Jan 2023 17:37:36 +0100</pubDate><guid>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</guid><description>&lt;h1 id="the-problem-it-solves">
+&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p>
+&lt;h1 id="language-modeling-requirements">
+ Language modeling requirements
+ &lt;a class="heading-link" href="#language-modeling-requirements">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;p>The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic &lt;a href="ttps://d2l.ai/chapter_attention-mechanisms-and-transformers/transformer.html" >Transformer&lt;/a>, it has proven to be extremely effective. The reason for its success is explored in the paper &amp;ldquo;&lt;a href="https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html" class="external-link" target="_blank" rel="noopener">In-Context Learning and Induction Heads&lt;/a>.&amp;rdquo; The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:&lt;/p></description></item><item><title>Paper overview: Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations</title><link>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</link><pubDate>Mon, 16 Jan 2023 17:37:36 +0100</pubDate><guid>https://n1o.github.io/posts/continuous-time-modeling-of-counterfatual-outcomes/</guid><description>&lt;h1 id="the-problem-it-solves">
  The problem it solves
  &lt;a class="heading-link" href="#the-problem-it-solves">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
@@ -68,7 +77,22 @@ However, it does have a limitation - its context length is restricted to 512 tok
  &lt;/a>
 &lt;/h1>
 &lt;p>Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:&lt;/p>
-&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p></description></item><item><title>Hierarchical Probabilistic Matrix Factorization</title><link>https://n1o.github.io/posts/pooled_matrix_factorization/</link><pubDate>Thu, 17 Dec 2020 10:19:42 +0100</pubDate><guid>https://n1o.github.io/posts/pooled_matrix_factorization/</guid><description>&lt;p>Probabilistic Matrix factorization is a simple but useful model for matrix imputation. The main idea is to decompose a tall and wide matrix into a product of two matrices, one tall and thin and one short and wide.&lt;/p>
+&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p>
+&lt;h1 id="an-example-and-some-details">
+ An example and some details
+ &lt;a class="heading-link" href="#an-example-and-some-details">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;h2 id="example">
+ Example
+ &lt;a class="heading-link" href="#example">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h2>
+&lt;p>As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient&amp;rsquo;s CRP levels again.&lt;/p></description></item><item><title>Hierarchical Probabilistic Matrix Factorization</title><link>https://n1o.github.io/posts/pooled_matrix_factorization/</link><pubDate>Thu, 17 Dec 2020 10:19:42 +0100</pubDate><guid>https://n1o.github.io/posts/pooled_matrix_factorization/</guid><description>&lt;p>Probabilistic Matrix factorization is a simple but useful model for matrix imputation. The main idea is to decompose a tall and wide matrix into a product of two matrices, one tall and thin and one short and wide.&lt;/p>
 &lt;p>$$
 R_{n\times m} = U_{m \times d} \cdot V_{d \times n}
 $$&lt;/p>
diff --git a/posts/t5-and-reverse-engineering/index.html b/posts/t5-and-reverse-engineering/index.html
index 04e9203..842a9b8 100644
--- a/posts/t5-and-reverse-engineering/index.html
+++ b/posts/t5-and-reverse-engineering/index.html
@@ -7,7 +7,10 @@
   
 
 For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create CodeBreakers
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="BinT5 and HexT5 or T5 and Binary Reverse Engineering"><meta name=twitter:description content=" Abstract Link to heading For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create CodeBreakers a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications."><meta property="og:url" content="https://n1o.github.io/posts/t5-and-reverse-engineering/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="BinT5 and HexT5 or T5 and Binary Reverse Engineering"><meta property="og:description" content=" Abstract Link to heading For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create CodeBreakers a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications."><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2024-06-12T14:07:17+02:00"><meta property="article:modified_time" content="2024-06-12T14:07:17+02:00"><meta property="article:tag" content="NLP"><meta property="article:tag" content="T5"><meta property="article:tag" content="Transformers"><link rel=canonical href=https://n1o.github.io/posts/t5-and-reverse-engineering/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.
+I found two notable research papers where T5 has been applied to reverse engineering are BinT5 and HexT5. Before we dive deep into the details of these papers, let&rsquo;s first explore the basics of reverse engineering."><meta name=keywords content="blog,developer,personal"><meta name=fediverse:creator content><meta name=twitter:card content="summary"><meta name=twitter:title content="BinT5 and HexT5 or T5 and Binary Reverse Engineering"><meta name=twitter:description content=" Abstract Link to heading For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create CodeBreakers a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.
+I found two notable research papers where T5 has been applied to reverse engineering are BinT5 and HexT5. Before we dive deep into the details of these papers, let’s first explore the basics of reverse engineering."><meta property="og:url" content="https://n1o.github.io/posts/t5-and-reverse-engineering/"><meta property="og:site_name" content="Data, Code and Breaking Stuff"><meta property="og:title" content="BinT5 and HexT5 or T5 and Binary Reverse Engineering"><meta property="og:description" content=" Abstract Link to heading For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create CodeBreakers a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.
+I found two notable research papers where T5 has been applied to reverse engineering are BinT5 and HexT5. Before we dive deep into the details of these papers, let’s first explore the basics of reverse engineering."><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="posts"><meta property="article:published_time" content="2024-06-12T14:07:17+02:00"><meta property="article:modified_time" content="2024-06-12T14:07:17+02:00"><meta property="article:tag" content="NLP"><meta property="article:tag" content="T5"><meta property="article:tag" content="Transformers"><link rel=canonical href=https://n1o.github.io/posts/t5-and-reverse-engineering/><link rel=preload href=/fonts/fa-brands-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-regular-400.woff2 as=font type=font/woff2 crossorigin><link rel=preload href=/fonts/fa-solid-900.woff2 as=font type=font/woff2 crossorigin><link rel=stylesheet href=/css/coder.min.e927f7340e309d76dcb8fda85f1531ae7341aa9cd0b7f3ab77885dae77b1a0a2.css integrity="sha256-6Sf3NA4wnXbcuP2oXxUxrnNBqpzQt/Ord4hdrnexoKI=" crossorigin=anonymous media=screen><link rel=stylesheet href=/css/coder-dark.min.a00e6364bacbc8266ad1cc81230774a1397198f8cfb7bcba29b7d6fcb54ce57f.css integrity="sha256-oA5jZLrLyCZq0cyBIwd0oTlxmPjPt7y6KbfW/LVM5X8=" crossorigin=anonymous media=screen><link rel=icon type=image/svg+xml href=/images/favicon.svg sizes=any><link rel=icon type=image/png href=/images/favicon-32x32.png sizes=32x32><link rel=icon type=image/png href=/images/favicon-16x16.png sizes=16x16><link rel=apple-touch-icon href=/images/apple-touch-icon.png><link rel=apple-touch-icon sizes=180x180 href=/images/apple-touch-icon.png><link rel=manifest href=/site.webmanifest><link rel=mask-icon href=/images/safari-pinned-tab.svg color=#5bbad5></head><body class="preload-transitions colorscheme-auto"><div class=float-container><a id=dark-mode-toggle class=colorscheme-toggle><i class="fa-solid fa-adjust fa-fw" aria-hidden=true></i></a></div><main class=wrapper><nav class=navigation><section class=container><a class=navigation-title href=https://n1o.github.io/>Data, Code and Breaking Stuff
 </a><input type=checkbox id=menu-toggle>
 <label class="menu-button float-right" for=menu-toggle><i class="fa-solid fa-bars fa-fw" aria-hidden=true></i></label><ul class=navigation-list><li class=navigation-item><a class=navigation-link href=/about/>About</a></li><li class=navigation-item><a class=navigation-link href=/posts/>Blog</a></li><li class=navigation-item><a class=navigation-link href=/awesome-t5/>Awesome T5</a></li><li class=navigation-item><a class=navigation-link href=/awesome-ssm/>Awesome SSM</a></li><li class=navigation-item><a class=navigation-link href=/projects/>Projects</a></li><li class=navigation-item><a class=navigation-link href=/contact/>Contact me</a></li></ul></section></nav><div class=content><section class="container post"><article><header><div class=post-title><h1 class=title><a class=title-link href=https://n1o.github.io/posts/t5-and-reverse-engineering/>BinT5 and HexT5 or T5 and Binary Reverse Engineering</a></h1></div><div class=post-meta><div class=date><span class=posted-on><i class="fa-solid fa-calendar" aria-hidden=true></i>
 <time datetime=2024-06-12T14:07:17+02:00>June 12, 2024
diff --git a/tags/causality/index.xml b/tags/causality/index.xml
index d20d311..215a6d3 100644
--- a/tags/causality/index.xml
+++ b/tags/causality/index.xml
@@ -6,4 +6,19 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:&lt;/p>
-&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p></description></item></channel></rss>
\ No newline at end of file
+&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p>
+&lt;h1 id="an-example-and-some-details">
+ An example and some details
+ &lt;a class="heading-link" href="#an-example-and-some-details">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;h2 id="example">
+ Example
+ &lt;a class="heading-link" href="#example">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h2>
+&lt;p>As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient&amp;rsquo;s CRP levels again.&lt;/p></description></item></channel></rss>
\ No newline at end of file
diff --git a/tags/differential_equation/index.xml b/tags/differential_equation/index.xml
index aff43a1..cc13df3 100644
--- a/tags/differential_equation/index.xml
+++ b/tags/differential_equation/index.xml
@@ -6,4 +6,19 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>Imagine you have an irregullary sampled time series, where at various time points we perform interventions. These interventions may influence the dynamics of the timeseries. The question we want to answer is:&lt;/p>
-&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p></description></item></channel></rss>
\ No newline at end of file
+&lt;p>&lt;em>If I perform a hypothetical sequence of interventions how will my time series evolve?&lt;/em>&lt;/p>
+&lt;h1 id="an-example-and-some-details">
+ An example and some details
+ &lt;a class="heading-link" href="#an-example-and-some-details">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;h2 id="example">
+ Example
+ &lt;a class="heading-link" href="#example">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h2>
+&lt;p>As a medical doctor in a hospital, if a patient has a high fever and is at risk of dying, a common practice is to measure their levels of C-reactive protein (CRP) to determine if they need antibiotics. If the CRP level is high, antibiotics are administered at a certain dosage. However, the effects of antibiotics may not be immediate and it may take some time before they show. In this scenario, the doctor would wait for some time and measure the patient&amp;rsquo;s CRP levels again.&lt;/p></description></item></channel></rss>
\ No newline at end of file
diff --git a/tags/nlp/index.xml b/tags/nlp/index.xml
index b750c76..50afbb0 100644
--- a/tags/nlp/index.xml
+++ b/tags/nlp/index.xml
@@ -27,7 +27,8 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create &lt;a href="https://codebreakers.re" class="external-link" target="_blank" rel="noopener">CodeBreakers&lt;/a>
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p>
+&lt;p>I found two notable research papers where &lt;a href="https://n1o.github.io/posts/t5-the-old-new-thing/" >T5&lt;/a> has been applied to reverse engineering are &lt;a href="https://arxiv.org/abs/2301.01701" class="external-link" target="_blank" rel="noopener">BinT5&lt;/a> and &lt;a href="https://www.semanticscholar.org/paper/HexT5%3A-Unified-Pre-Training-for-Stripped-Binary-Xiong-Chen/04c3fccfe01f42afe18dcdb027385f350ab3c9d1" class="external-link" target="_blank" rel="noopener">HexT5&lt;/a>. Before we dive deep into the details of these papers, let&amp;rsquo;s first explore the basics of reverse engineering.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
  Abstract
  &lt;a class="heading-link" href="#abstract">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
@@ -59,4 +60,12 @@ However, it does have a limitation - its context length is restricted to 512 tok
  &lt;span class="sr-only">Link to heading&lt;/span>
  &lt;/a>
 &lt;/h1>
-&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p></description></item></channel></rss>
\ No newline at end of file
+&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p>
+&lt;h1 id="language-modeling-requirements">
+ Language modeling requirements
+ &lt;a class="heading-link" href="#language-modeling-requirements">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;p>The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic &lt;a href="ttps://d2l.ai/chapter_attention-mechanisms-and-transformers/transformer.html" >Transformer&lt;/a>, it has proven to be extremely effective. The reason for its success is explored in the paper &amp;ldquo;&lt;a href="https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html" class="external-link" target="_blank" rel="noopener">In-Context Learning and Induction Heads&lt;/a>.&amp;rdquo; The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:&lt;/p></description></item></channel></rss>
\ No newline at end of file
diff --git a/tags/ssm/index.xml b/tags/ssm/index.xml
index a9227e0..29e260c 100644
--- a/tags/ssm/index.xml
+++ b/tags/ssm/index.xml
@@ -26,4 +26,12 @@
  &lt;span class="sr-only">Link to heading&lt;/span>
  &lt;/a>
 &lt;/h1>
-&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p></description></item></channel></rss>
\ No newline at end of file
+&lt;p>By combining State Space Models (SSMs) with Attention, we get a model that generates text more efficiently, with a speed increase of approximately 1.6 times. Additionally, this approach requires less paremters, enabling the development of larger models on existing hardware.&lt;/p>
+&lt;h1 id="language-modeling-requirements">
+ Language modeling requirements
+ &lt;a class="heading-link" href="#language-modeling-requirements">
+ &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
+ &lt;span class="sr-only">Link to heading&lt;/span>
+ &lt;/a>
+&lt;/h1>
+&lt;p>The Transformer architecture, which forms the basis of ChatGPT, is riding high on the hype train due to its impressive performance. Although it is just a basic &lt;a href="ttps://d2l.ai/chapter_attention-mechanisms-and-transformers/transformer.html" >Transformer&lt;/a>, it has proven to be extremely effective. The reason for its success is explored in the paper &amp;ldquo;&lt;a href="https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html" class="external-link" target="_blank" rel="noopener">In-Context Learning and Induction Heads&lt;/a>.&amp;rdquo; The authors argue that the majority of the in-context learning capacity of the Transformer architecture can be evaluated by two tests:&lt;/p></description></item></channel></rss>
\ No newline at end of file
diff --git a/tags/t5/index.xml b/tags/t5/index.xml
index 9443792..791ee26 100644
--- a/tags/t5/index.xml
+++ b/tags/t5/index.xml
@@ -6,7 +6,8 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create &lt;a href="https://codebreakers.re" class="external-link" target="_blank" rel="noopener">CodeBreakers&lt;/a>
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p>
+&lt;p>I found two notable research papers where &lt;a href="https://n1o.github.io/posts/t5-the-old-new-thing/" >T5&lt;/a> has been applied to reverse engineering are &lt;a href="https://arxiv.org/abs/2301.01701" class="external-link" target="_blank" rel="noopener">BinT5&lt;/a> and &lt;a href="https://www.semanticscholar.org/paper/HexT5%3A-Unified-Pre-Training-for-Stripped-Binary-Xiong-Chen/04c3fccfe01f42afe18dcdb027385f350ab3c9d1" class="external-link" target="_blank" rel="noopener">HexT5&lt;/a>. Before we dive deep into the details of these papers, let&amp;rsquo;s first explore the basics of reverse engineering.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
  Abstract
  &lt;a class="heading-link" href="#abstract">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>
diff --git a/tags/transformers/index.xml b/tags/transformers/index.xml
index 31bf278..4d8a0ec 100644
--- a/tags/transformers/index.xml
+++ b/tags/transformers/index.xml
@@ -6,7 +6,8 @@
  &lt;/a>
 &lt;/h1>
 &lt;p>For a while now I have a new passion and that is binary reverse engineering and vulnerability exploitation. This interest has led me to create &lt;a href="https://codebreakers.re" class="external-link" target="_blank" rel="noopener">CodeBreakers&lt;/a>
-a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
+a platform dedicated to applying machine learning to reverse engineering, vulnerability detection, exploitation, and other cybersecurity-related applications.&lt;/p>
+&lt;p>I found two notable research papers where &lt;a href="https://n1o.github.io/posts/t5-the-old-new-thing/" >T5&lt;/a> has been applied to reverse engineering are &lt;a href="https://arxiv.org/abs/2301.01701" class="external-link" target="_blank" rel="noopener">BinT5&lt;/a> and &lt;a href="https://www.semanticscholar.org/paper/HexT5%3A-Unified-Pre-Training-for-Stripped-Binary-Xiong-Chen/04c3fccfe01f42afe18dcdb027385f350ab3c9d1" class="external-link" target="_blank" rel="noopener">HexT5&lt;/a>. Before we dive deep into the details of these papers, let&amp;rsquo;s first explore the basics of reverse engineering.&lt;/p></description></item><item><title>CodeT5 and CodeT5+</title><link>https://n1o.github.io/posts/code-t5-plus/</link><pubDate>Sat, 01 Jun 2024 08:46:43 +0200</pubDate><guid>https://n1o.github.io/posts/code-t5-plus/</guid><description>&lt;h1 id="abstract">
  Abstract
  &lt;a class="heading-link" href="#abstract">
  &lt;i class="fa-solid fa-link" aria-hidden="true" title="Link to heading">&lt;/i>