diff --git a/.hugo_build.lock b/.hugo_build.lock new file mode 100644 index 0000000..e69de29 diff --git a/config.toml b/config.toml index b0ecfb7..6858e91 100644 --- a/config.toml +++ b/config.toml @@ -110,11 +110,16 @@ disqusShortname = "mbarak-io" url = "posts/" [[languages.en.menu.main]] - name = "Projects" + name = "Awesome T5" weight = 3 + url = "awesome-t5/" + + [[languages.en.menu.main]] + name = "Projects" + weight = 4 url = "projects/" [[languages.en.menu.main]] name = "Contact me" - weight = 4 + weight = 5 url = "contact/" diff --git a/content/awesome-t5.md b/content/awesome-t5.md new file mode 100644 index 0000000..9eb34d6 --- /dev/null +++ b/content/awesome-t5.md @@ -0,0 +1,24 @@ +--- +title: "Awesome T5" +date: 2024-05-06T17:31:29+02:00 +draft: true +--- + +Here is a bunch of my posts about a super cool model called T5 its extensions and applications. + +# Basics +- [T5, FLAN-T5, UL2]({{< relref "posts/t5-the-old-new-thing.md" >}}) +- [LongT5, CoLT5]({{< relref "posts/longer-context-for-t5.md" >}}) + + +# Coding T5 +- CodeT5, CodeT5+ +- AST-T5 +- CodeFusion + +# Cyber Security +- LENS + +# With Graphs +- Graph Neural Prompting + diff --git a/content/posts/longer-context-for-t5.md b/content/posts/longer-context-for-t5.md new file mode 100644 index 0000000..78ed71e --- /dev/null +++ b/content/posts/longer-context-for-t5.md @@ -0,0 +1,60 @@ +--- +title: "Longer Context for T5" +date: 2024-04-29T14:10:36+02:00 +draft: true +--- + +# Why does T5 need a longer context? + +In my previous post +[T5 the Old New Thing]({{< relref "posts/t5-the-old-new-thing.md" >}}) +we already explored why T5 is awesome. But one downside is its limited context length of 512 tokens. Now technically this cannot be compared to the context length of an decoder-only model, since T5 is an encoder-decoder model, meaning the encoder can process input up to 512 tokens, and the decoder can generate output up to 512 tokens. Making the total context length 1024 tokens. We will cover two extensions: +1. [LongT5](https://arxiv.org/abs/2112.07916) +2. [ColtT5](https://arxiv.org/abs/2303.09752) + +Booth LongT5 and CoLT5 are eploring ways how to extend the context length of the encoder part of T5. This means that we are looking into how we are able to process longer input lengths and not necessarily generate longer texts. This approach is especially useful for problems like text summarization or document question answering. + +# LongT5 + +Originally published in 2022 and it uses a new pretraining strategy called Pegasus and explores using Local Attention and a novel TGlobal attention in the encoder. + +## Pegasus +Pegasus is a pretraining strategy pecially designed for abstract summarization, where we mask out key (principle) sentecnes from a document and we ask the model to reproduce them as a single string as if it would be a summary. + +## Local Attention +This is just a sliding window attention, thus any given token is able to attend to a neighborhood of $l$ tokens. + +## TGlobal +In TGlobal we partition the input tokens into chucks of length $k$, for each chuck we compute a global token by summing up the individual token embeddings. Now when we perform attention we take a Local Window $l$ and we apped to it all the global tokens. + +![TGlobal](/images/t_global_attention.png) + +### Cons +Since we do not perform full attention we experience some minor performance degradation, and we need an couple of extra parameters. Computation wise we compute the global tokens on the fly, but we can compute them only once per input tokens per layer and cache them. + +### Pros +We are able to process way larger input lengths. + +## Notes +It is worth noting that there is a variant of LongT5 that uses only Local Attention without Global Tokens. This wariant can be scaled up to even longer sequences unfortunately there is also an more profound performance drop. + +# CoLT5 +Paper from 2023, and it builds upon LongT5 by bring in ideas like [Mixture of Experts](https://arxiv.org/abs/2101.03961) and [Multi Query Attention](https://arxiv.org/abs/1911.02150). + +## Conditional Computation + +![ColT5 Attention](/images/colt5_transformer_layer.png) + +### Routing + +## Decoder +During output generation long input setences cause an memory bandwidth bottle neck, by using Multi Query Attention (MQA) to speed up the decoding process. In MQA all the query heads share the same key-value pair. + +![Grouped Attention Variants](/images/klu-gqa-grouped-query-attention.webp) + +### Performance +Vanilla Multi Head Attention tends to have the highest accuracy but it requires the most memory and it is the slowest to generate, by allowing query heads to share key-value pairs we decrease the memory requirements and improve token generation speed. Unfortunately this speedup comes at a cost and we loose accuracy. + + +## Cons +Where with LongT5 we have an open source implementation at hug diff --git a/content/posts/t5-the-old-new-thing.md b/content/posts/t5-the-old-new-thing.md index 949d320..0072d69 100644 --- a/content/posts/t5-the-old-new-thing.md +++ b/content/posts/t5-the-old-new-thing.md @@ -129,3 +129,4 @@ Furthermore, T5 boasts runtime requirements that are approximately half those of # Disclaimer Since I am not an english native speaker, I use ChatGPT to help me with the text (Formatting, Spelling, etc). However I did write every single word in this blog post, If you are interested you can check the the original text [here](https://github.com/n1o/n1o.github.io/blob/master/content/posts/t5-the-old-new-thing.md) + diff --git a/resources/_gen/assets/scss/scss/coder-dark.scss_9e20ccd2d8034c8e0fd83b11fb6e2bd5.content b/resources/_gen/assets/scss/scss/coder-dark.scss_9e20ccd2d8034c8e0fd83b11fb6e2bd5.content index 4299b45..0822dac 100644 --- a/resources/_gen/assets/scss/scss/coder-dark.scss_9e20ccd2d8034c8e0fd83b11fb6e2bd5.content +++ b/resources/_gen/assets/scss/scss/coder-dark.scss_9e20ccd2d8034c8e0fd83b11fb6e2bd5.content @@ -10,16 +10,65 @@ body.colorscheme-dark { body.colorscheme-dark h5, body.colorscheme-dark h6 { color: #dadada; } + body.colorscheme-dark h1:hover .heading-link, + body.colorscheme-dark h2:hover .heading-link, + body.colorscheme-dark h3:hover .heading-link, + body.colorscheme-dark h4:hover .heading-link, + body.colorscheme-dark h5:hover .heading-link, + body.colorscheme-dark h6:hover .heading-link { + visibility: visible; } + body.colorscheme-dark h1 .heading-link, + body.colorscheme-dark h2 .heading-link, + body.colorscheme-dark h3 .heading-link, + body.colorscheme-dark h4 .heading-link, + body.colorscheme-dark h5 .heading-link, + body.colorscheme-dark h6 .heading-link { + color: #42a5f5; + font-weight: inherit; + text-decoration: none; + font-size: 80%; + visibility: hidden; } + body.colorscheme-dark h1 .title-link, + body.colorscheme-dark h2 .title-link, + body.colorscheme-dark h3 .title-link, + body.colorscheme-dark h4 .title-link, + body.colorscheme-dark h5 .title-link, + body.colorscheme-dark h6 .title-link { + color: inherit; + font-weight: inherit; + text-decoration: none; } body.colorscheme-dark code { background-color: #424242; color: #dadada; } - body.colorscheme-dark pre code { + body.colorscheme-dark .highlight pre { + background-color: #424242; + color: #212121; } + body.colorscheme-dark .highlight pre code { + background-color: inherit; + color: inherit; } + body.colorscheme-dark :not(.highlight) > pre code { background-color: inherit; color: inherit; } body.colorscheme-dark blockquote { border-left: 2px solid #424242; } - body.colorscheme-dark table td, body.colorscheme-dark table th { + body.colorscheme-dark th, + body.colorscheme-dark td { + padding: 1.6rem; } + body.colorscheme-dark table { + border-collapse: collapse; } + body.colorscheme-dark table td, + body.colorscheme-dark table th { border: 2px solid #dadada; } + body.colorscheme-dark table tr:first-child th { + border-top: 0; } + body.colorscheme-dark table tr:last-child td { + border-bottom: 0; } + body.colorscheme-dark table tr td:first-child, + body.colorscheme-dark table tr th:first-child { + border-left: 0; } + body.colorscheme-dark table tr td:last-child, + body.colorscheme-dark table tr th:last-child { + border-right: 0; } @media (prefers-color-scheme: dark) { body.colorscheme-auto { @@ -34,16 +83,72 @@ body.colorscheme-dark { body.colorscheme-auto h5, body.colorscheme-auto h6 { color: #dadada; } + body.colorscheme-auto h1:hover .heading-link, + body.colorscheme-auto h2:hover .heading-link, + body.colorscheme-auto h3:hover .heading-link, + body.colorscheme-auto h4:hover .heading-link, + body.colorscheme-auto h5:hover .heading-link, + body.colorscheme-auto h6:hover .heading-link { + visibility: visible; } + body.colorscheme-auto h1 .heading-link, + body.colorscheme-auto h2 .heading-link, + body.colorscheme-auto h3 .heading-link, + body.colorscheme-auto h4 .heading-link, + body.colorscheme-auto h5 .heading-link, + body.colorscheme-auto h6 .heading-link { + color: #42a5f5; + font-weight: inherit; + text-decoration: none; + font-size: 80%; + visibility: hidden; } + body.colorscheme-auto h1 .title-link, + body.colorscheme-auto h2 .title-link, + body.colorscheme-auto h3 .title-link, + body.colorscheme-auto h4 .title-link, + body.colorscheme-auto h5 .title-link, + body.colorscheme-auto h6 .title-link { + color: inherit; + font-weight: inherit; + text-decoration: none; } body.colorscheme-auto code { background-color: #424242; color: #dadada; } - body.colorscheme-auto pre code { + body.colorscheme-auto .highlight pre { + background-color: #424242; + color: #212121; } + body.colorscheme-auto .highlight pre code { + background-color: inherit; + color: inherit; } + body.colorscheme-auto :not(.highlight) > pre code { background-color: inherit; color: inherit; } body.colorscheme-auto blockquote { border-left: 2px solid #424242; } - body.colorscheme-auto table td, body.colorscheme-auto table th { - border: 2px solid #dadada; } } + body.colorscheme-auto th, + body.colorscheme-auto td { + padding: 1.6rem; } + body.colorscheme-auto table { + border-collapse: collapse; } + body.colorscheme-auto table td, + body.colorscheme-auto table th { + border: 2px solid #dadada; } + body.colorscheme-auto table tr:first-child th { + border-top: 0; } + body.colorscheme-auto table tr:last-child td { + border-bottom: 0; } + body.colorscheme-auto table tr td:first-child, + body.colorscheme-auto table tr th:first-child { + border-left: 0; } + body.colorscheme-auto table tr td:last-child, + body.colorscheme-auto table tr th:last-child { + border-right: 0; } } + +body.colorscheme-dark .content .post .tags .tag { + background-color: #424242; } + body.colorscheme-dark .content .post .tags .tag a { + color: #dadada; } + body.colorscheme-dark .content .post .tags .tag a:active { + color: #dadada; } body.colorscheme-dark .content .list ul li .title { color: #dadada; } @@ -56,6 +161,12 @@ body.colorscheme-dark .content .centered .about ul li a { color: #42a5f5; } @media (prefers-color-scheme: dark) { + body.colorscheme-auto .content .post .tags .tag { + background-color: #424242; } + body.colorscheme-auto .content .post .tags .tag a { + color: #dadada; } + body.colorscheme-auto .content .post .tags .tag a:active { + color: #dadada; } body.colorscheme-auto .content .list ul li .title { color: #dadada; } body.colorscheme-auto .content .list ul li .title:hover, body.colorscheme-auto .content .list ul li .title:focus { @@ -65,7 +176,15 @@ body.colorscheme-dark .content .centered .about ul li a { body.colorscheme-auto .content .centered .about ul li a:hover, body.colorscheme-auto .content .centered .about ul li a:focus { color: #42a5f5; } } -body.colorscheme-dark .navigation a, body.colorscheme-dark .navigation span { +body.colorscheme-dark .notice .notice-title { + border-bottom: 1px solid #212121; } + +@media (prefers-color-scheme: dark) { + body.colorscheme-auto .notice .notice-title { + border-bottom: 1px solid #212121; } } + +body.colorscheme-dark .navigation a, +body.colorscheme-dark .navigation span { color: #dadada; } body.colorscheme-dark .navigation a:hover, body.colorscheme-dark .navigation a:focus { @@ -94,7 +213,8 @@ body.colorscheme-dark .navigation .menu-button i:hover, body.colorscheme-dark .n color: #dadada; } @media (prefers-color-scheme: dark) { - body.colorscheme-auto .navigation a, body.colorscheme-auto .navigation span { + body.colorscheme-auto .navigation a, + body.colorscheme-auto .navigation span { color: #dadada; } body.colorscheme-auto .navigation a:hover, body.colorscheme-auto .navigation a:focus { color: #42a5f5; } } @@ -121,6 +241,42 @@ body.colorscheme-dark .navigation .menu-button i:hover, body.colorscheme-dark .n body.colorscheme-auto .navigation .menu-button i:hover, body.colorscheme-auto .navigation .menu-button i:focus { color: #dadada; } } +body.colorscheme-dark .tabs label.tab-label { + background-color: #424242; + border-color: #4f4f4f; } + +body.colorscheme-dark .tabs input.tab-input:checked + label.tab-label { + background-color: #212121; } + +body.colorscheme-dark .tabs .tab-content { + background-color: #212121; + border-color: #4f4f4f; } + +@media (prefers-color-scheme: dark) { + body.colorscheme-auto .tabs label.tab-label { + background-color: #424242; + border-color: #4f4f4f; } + body.colorscheme-auto .tabs input.tab-input:checked + label.tab-label { + background-color: #212121; } + body.colorscheme-auto .tabs .tab-content { + background-color: #212121; + border-color: #4f4f4f; } } + +body.colorscheme-dark .taxonomy-element { + background-color: #424242; } + body.colorscheme-dark .taxonomy-element a { + color: #dadada; } + body.colorscheme-dark .taxonomy-element a:active { + color: #dadada; } + +@media (prefers-color-scheme: dark) { + body.colorscheme-auto .taxonomy-element { + background-color: #424242; } + body.colorscheme-auto .taxonomy-element a { + color: #dadada; } + body.colorscheme-auto .taxonomy-element a:active { + color: #dadada; } } + body.colorscheme-dark .footer a { color: #42a5f5; } diff --git a/resources/_gen/assets/scss/scss/coder.scss_fd4b5b3f9a48bc0c7f005d2f7a4cc30f.content b/resources/_gen/assets/scss/scss/coder.scss_fd4b5b3f9a48bc0c7f005d2f7a4cc30f.content index 66304d9..e7aea9e 100644 --- a/resources/_gen/assets/scss/scss/coder.scss_fd4b5b3f9a48bc0c7f005d2f7a4cc30f.content +++ b/resources/_gen/assets/scss/scss/coder.scss_fd4b5b3f9a48bc0c7f005d2f7a4cc30f.content @@ -1,3 +1,3035 @@ +@charset "UTF-8"; +/*! normalize.css v8.0.1 | MIT License | github.com/necolas/normalize.css */ +/* Document + ========================================================================== */ +/** + * 1. Correct the line height in all browsers. + * 2. Prevent adjustments of font size after orientation changes in iOS. + */ +html { + line-height: 1.15; + /* 1 */ + -webkit-text-size-adjust: 100%; + /* 2 */ } + +/* Sections + ========================================================================== */ +/** + * Remove the margin in all browsers. + */ +body { + margin: 0; } + +/** + * Render the `main` element consistently in IE. + */ +main { + display: block; } + +/** + * Correct the font size and margin on `h1` elements within `section` and + * `article` contexts in Chrome, Firefox, and Safari. + */ +h1 { + font-size: 2em; + margin: 0.67em 0; } + +/* Grouping content + ========================================================================== */ +/** + * 1. Add the correct box sizing in Firefox. + * 2. Show the overflow in Edge and IE. + */ +hr { + box-sizing: content-box; + /* 1 */ + height: 0; + /* 1 */ + overflow: visible; + /* 2 */ } + +/** + * 1. Correct the inheritance and scaling of font size in all browsers. + * 2. Correct the odd `em` font sizing in all browsers. + */ +pre { + font-family: monospace, monospace; + /* 1 */ + font-size: 1em; + /* 2 */ } + +/* Text-level semantics + ========================================================================== */ +/** + * Remove the gray background on active links in IE 10. + */ +a { + background-color: transparent; + word-wrap: break-word; } + +/** + * 1. Remove the bottom border in Chrome 57- + * 2. Add the correct text decoration in Chrome, Edge, IE, Opera, and Safari. + */ +abbr[title] { + border-bottom: none; + /* 1 */ + text-decoration: underline; + /* 2 */ + text-decoration: underline dotted; + /* 2 */ } + +/** + * Add the correct font weight in Chrome, Edge, and Safari. + */ +b, +strong { + font-weight: bolder; } + +/** + * 1. Correct the inheritance and scaling of font size in all browsers. + * 2. Correct the odd `em` font sizing in all browsers. + */ +code, +kbd, +samp { + font-family: monospace, monospace; + /* 1 */ + font-size: 1em; + /* 2 */ } + +/** + * Add the correct font size in all browsers. + */ +small { + font-size: 80%; } + +/** + * Prevent `sub` and `sup` elements from affecting the line height in + * all browsers. + */ +sub, +sup { + font-size: 75%; + line-height: 0; + position: relative; + vertical-align: baseline; } + +sub { + bottom: -0.25em; } + +sup { + top: -0.5em; } + +/* Embedded content + ========================================================================== */ +/** + * Remove the border on images inside links in IE 10. + */ +img { + border-style: none; } + +/* Forms + ========================================================================== */ +/** + * 1. Change the font styles in all browsers. + * 2. Remove the margin in Firefox and Safari. + */ +button, +input, +optgroup, +select, +textarea { + font-family: inherit; + /* 1 */ + font-size: 100%; + /* 1 */ + line-height: 1.15; + /* 1 */ + margin: 0; + /* 2 */ } + +/** + * Show the overflow in IE. + * 1. Show the overflow in Edge. + */ +button, +input { + /* 1 */ + overflow: visible; } + +/** + * Remove the inheritance of text transform in Edge, Firefox, and IE. + * 1. Remove the inheritance of text transform in Firefox. + */ +button, +select { + /* 1 */ + text-transform: none; } + +/** + * Correct the inability to style clickable types in iOS and Safari. + */ +button, +[type="button"], +[type="reset"], +[type="submit"] { + -webkit-appearance: button; } + +/** + * Remove the inner border and padding in Firefox. + */ +button::-moz-focus-inner, +[type="button"]::-moz-focus-inner, +[type="reset"]::-moz-focus-inner, +[type="submit"]::-moz-focus-inner { + border-style: none; + padding: 0; } + +/** + * Restore the focus styles unset by the previous rule. + */ +button:-moz-focusring, +[type="button"]:-moz-focusring, +[type="reset"]:-moz-focusring, +[type="submit"]:-moz-focusring { + outline: 1px dotted ButtonText; } + +/** + * Correct the padding in Firefox. + */ +fieldset { + padding: 0.35em 0.75em 0.625em; } + +/** + * 1. Correct the text wrapping in Edge and IE. + * 2. Correct the color inheritance from `fieldset` elements in IE. + * 3. Remove the padding so developers are not caught out when they zero out + * `fieldset` elements in all browsers. + */ +legend { + box-sizing: border-box; + /* 1 */ + color: inherit; + /* 2 */ + display: table; + /* 1 */ + max-width: 100%; + /* 1 */ + padding: 0; + /* 3 */ + white-space: normal; + /* 1 */ } + +/** + * Add the correct vertical alignment in Chrome, Firefox, and Opera. + */ +progress { + vertical-align: baseline; } + +/** + * Remove the default vertical scrollbar in IE 10+. + */ +textarea { + overflow: auto; } + +/** + * 1. Add the correct box sizing in IE 10. + * 2. Remove the padding in IE 10. + */ +[type="checkbox"], +[type="radio"] { + box-sizing: border-box; + /* 1 */ + padding: 0; + /* 2 */ } + +/** + * Correct the cursor style of increment and decrement buttons in Chrome. + */ +[type="number"]::-webkit-inner-spin-button, +[type="number"]::-webkit-outer-spin-button { + height: auto; } + +/** + * 1. Correct the odd appearance in Chrome and Safari. + * 2. Correct the outline style in Safari. + */ +[type="search"] { + -webkit-appearance: textfield; + /* 1 */ + outline-offset: -2px; + /* 2 */ } + +/** + * Remove the inner padding in Chrome and Safari on macOS. + */ +[type="search"]::-webkit-search-decoration { + -webkit-appearance: none; } + +/** + * 1. Correct the inability to style clickable types in iOS and Safari. + * 2. Change font properties to `inherit` in Safari. + */ +::-webkit-file-upload-button { + -webkit-appearance: button; + /* 1 */ + font: inherit; + /* 2 */ } + +/* Interactive + ========================================================================== */ +/* + * Add the correct display in Edge, IE 10+, and Firefox. + */ +details { + display: block; } + +/* + * Add the correct display in all browsers. + */ +summary { + display: list-item; } + +/* Misc + ========================================================================== */ +/** + * Add the correct display in IE 10+. + */ +template { + display: none; } + +/** + * Add the correct display in IE 10. + */ +[hidden] { + display: none; } + +/*! +Fork Awesome 1.2.0 +License - https://forkaweso.me/Fork-Awesome/license + +Copyright 2018 Dave Gandy & Fork Awesome + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +/* FONT PATH + * -------------------------- */ +@font-face { + font-family: "ForkAwesome"; + src: url("../fonts/forkawesome-webfont.eot?v=1.2.0"); + src: url("../fonts/forkawesome-webfont.eot?#iefix&v=1.2.0") format("embedded-opentype"), url("../fonts/forkawesome-webfont.woff2?v=1.2.0") format("woff2"), url("../fonts/forkawesome-webfont.woff?v=1.2.0") format("woff"), url("../fonts/forkawesome-webfont.ttf?v=1.2.0") format("truetype"), url("../fonts/forkawesome-webfont.svg?v=1.2.0#forkawesomeregular") format("svg"); + font-weight: normal; + font-style: normal; + font-display: block; } + +.fa { + display: inline-block; + font: normal normal normal 14px/1 ForkAwesome; + font-size: inherit; + text-rendering: auto; + -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; } + +/* makes the font 33% larger relative to the icon container */ +.fa-lg { + font-size: 1.33333333em; + line-height: 0.75em; + vertical-align: -15%; } + +.fa-2x { + font-size: 2em; } + +.fa-3x { + font-size: 3em; } + +.fa-4x { + font-size: 4em; } + +.fa-5x { + font-size: 5em; } + +.fa-fw { + width: 1.28571429em; + text-align: center; } + +.fa-ul { + padding-left: 0; + margin-left: 2.14285714em; + list-style-type: none; } + .fa-ul > li { + position: relative; } + +.fa-li { + position: absolute; + left: -2.14285714em; + width: 2.14285714em; + top: 0.14285714em; + text-align: center; } + .fa-li.fa-lg { + left: -1.85714286em; } + +.fa-border { + padding: .2em .25em .15em; + border: solid 0.08em #eee; + border-radius: .1em; } + +.fa-pull-left { + float: left; } + +.fa-pull-right { + float: right; } + +.fa.fa-pull-left { + margin-right: .3em; } + +.fa.fa-pull-right { + margin-left: .3em; } + +/* Deprecated as of 4.4.0 */ +.pull-right { + float: right; } + +.pull-left { + float: left; } + +.fa.pull-left { + margin-right: .3em; } + +.fa.pull-right { + margin-left: .3em; } + +.fa-spin { + -webkit-animation: fa-spin 2s infinite linear; + animation: fa-spin 2s infinite linear; } + +.fa-pulse { + -webkit-animation: fa-spin 1s infinite steps(8); + animation: fa-spin 1s infinite steps(8); } + +@-webkit-keyframes fa-spin { + 0% { + -webkit-transform: rotate(0deg); + transform: rotate(0deg); } + 100% { + -webkit-transform: rotate(359deg); + transform: rotate(359deg); } } + +@keyframes fa-spin { + 0% { + -webkit-transform: rotate(0deg); + transform: rotate(0deg); } + 100% { + -webkit-transform: rotate(359deg); + transform: rotate(359deg); } } + +.fa-rotate-90 { + -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=1)"; + -webkit-transform: rotate(90deg); + -ms-transform: rotate(90deg); + transform: rotate(90deg); } + +.fa-rotate-180 { + -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=2)"; + -webkit-transform: rotate(180deg); + -ms-transform: rotate(180deg); + transform: rotate(180deg); } + +.fa-rotate-270 { + -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=3)"; + -webkit-transform: rotate(270deg); + -ms-transform: rotate(270deg); + transform: rotate(270deg); } + +.fa-flip-horizontal { + -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1)"; + -webkit-transform: scale(-1, 1); + -ms-transform: scale(-1, 1); + transform: scale(-1, 1); } + +.fa-flip-vertical { + -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1)"; + -webkit-transform: scale(1, -1); + -ms-transform: scale(1, -1); + transform: scale(1, -1); } + +:root .fa-rotate-90, +:root .fa-rotate-180, +:root .fa-rotate-270, +:root .fa-flip-horizontal, +:root .fa-flip-vertical { + filter: none; } + +.fa-stack { + position: relative; + display: inline-block; + width: 2em; + height: 2em; + line-height: 2em; + vertical-align: middle; } + +.fa-stack-1x, .fa-stack-2x { + position: absolute; + left: 0; + width: 100%; + text-align: center; } + +.fa-stack-1x { + line-height: inherit; } + +.fa-stack-2x { + font-size: 2em; } + +.fa-inverse { + color: #fff; } + +/* Font Awesome uses the Unicode Private Use Area (PUA) to ensure screen + readers do not read off random characters that represent icons */ +.fa-glass:before { + content: "\f000"; } + +.fa-music:before { + content: "\f001"; } + +.fa-search:before { + content: "\f002"; } + +.fa-envelope-o:before { + content: "\f003"; } + +.fa-heart:before { + content: "\f004"; } + +.fa-star:before { + content: "\f005"; } + +.fa-star-o:before { + content: "\f006"; } + +.fa-user:before { + content: "\f007"; } + +.fa-film:before { + content: "\f008"; } + +.fa-th-large:before { + content: "\f009"; } + +.fa-th:before { + content: "\f00a"; } + +.fa-th-list:before { + content: "\f00b"; } + +.fa-check:before { + content: "\f00c"; } + +.fa-remove:before, +.fa-close:before, +.fa-times:before { + content: "\f00d"; } + +.fa-search-plus:before { + content: "\f00e"; } + +.fa-search-minus:before { + content: "\f010"; } + +.fa-power-off:before { + content: "\f011"; } + +.fa-signal:before { + content: "\f012"; } + +.fa-gear:before, +.fa-cog:before { + content: "\f013"; } + +.fa-trash-o:before { + content: "\f014"; } + +.fa-home:before { + content: "\f015"; } + +.fa-file-o:before { + content: "\f016"; } + +.fa-clock-o:before { + content: "\f017"; } + +.fa-road:before { + content: "\f018"; } + +.fa-download:before { + content: "\f019"; } + +.fa-arrow-circle-o-down:before { + content: "\f01a"; } + +.fa-arrow-circle-o-up:before { + content: "\f01b"; } + +.fa-inbox:before { + content: "\f01c"; } + +.fa-play-circle-o:before { + content: "\f01d"; } + +.fa-rotate-right:before, +.fa-repeat:before { + content: "\f01e"; } + +.fa-sync:before, +.fa-refresh:before { + content: "\f021"; } + +.fa-list-alt:before { + content: "\f022"; } + +.fa-lock:before { + content: "\f023"; } + +.fa-flag:before { + content: "\f024"; } + +.fa-headphones:before { + content: "\f025"; } + +.fa-volume-off:before { + content: "\f026"; } + +.fa-volume-down:before { + content: "\f027"; } + +.fa-volume-up:before { + content: "\f028"; } + +.fa-qrcode:before { + content: "\f029"; } + +.fa-barcode:before { + content: "\f02a"; } + +.fa-tag:before { + content: "\f02b"; } + +.fa-tags:before { + content: "\f02c"; } + +.fa-book:before { + content: "\f02d"; } + +.fa-bookmark:before { + content: "\f02e"; } + +.fa-print:before { + content: "\f02f"; } + +.fa-camera:before { + content: "\f030"; } + +.fa-font:before { + content: "\f031"; } + +.fa-bold:before { + content: "\f032"; } + +.fa-italic:before { + content: "\f033"; } + +.fa-text-height:before { + content: "\f034"; } + +.fa-text-width:before { + content: "\f035"; } + +.fa-align-left:before { + content: "\f036"; } + +.fa-align-center:before { + content: "\f037"; } + +.fa-align-right:before { + content: "\f038"; } + +.fa-align-justify:before { + content: "\f039"; } + +.fa-list:before { + content: "\f03a"; } + +.fa-dedent:before, +.fa-outdent:before { + content: "\f03b"; } + +.fa-indent:before { + content: "\f03c"; } + +.fa-video:before, +.fa-video-camera:before { + content: "\f03d"; } + +.fa-photo:before, +.fa-image:before, +.fa-picture-o:before { + content: "\f03e"; } + +.fa-pencil:before { + content: "\f040"; } + +.fa-map-marker:before { + content: "\f041"; } + +.fa-adjust:before { + content: "\f042"; } + +.fa-tint:before { + content: "\f043"; } + +.fa-edit:before, +.fa-pencil-square-o:before { + content: "\f044"; } + +.fa-share-square-o:before { + content: "\f045"; } + +.fa-check-square-o:before { + content: "\f046"; } + +.fa-arrows:before { + content: "\f047"; } + +.fa-step-backward:before { + content: "\f048"; } + +.fa-fast-backward:before { + content: "\f049"; } + +.fa-backward:before { + content: "\f04a"; } + +.fa-play:before { + content: "\f04b"; } + +.fa-pause:before { + content: "\f04c"; } + +.fa-stop:before { + content: "\f04d"; } + +.fa-forward:before { + content: "\f04e"; } + +.fa-fast-forward:before { + content: "\f050"; } + +.fa-step-forward:before { + content: "\f051"; } + +.fa-eject:before { + content: "\f052"; } + +.fa-chevron-left:before { + content: "\f053"; } + +.fa-chevron-right:before { + content: "\f054"; } + +.fa-plus-circle:before { + content: "\f055"; } + +.fa-minus-circle:before { + content: "\f056"; } + +.fa-times-circle:before { + content: "\f057"; } + +.fa-check-circle:before { + content: "\f058"; } + +.fa-question-circle:before { + content: "\f059"; } + +.fa-info-circle:before { + content: "\f05a"; } + +.fa-crosshairs:before { + content: "\f05b"; } + +.fa-times-circle-o:before { + content: "\f05c"; } + +.fa-check-circle-o:before { + content: "\f05d"; } + +.fa-ban:before { + content: "\f05e"; } + +.fa-arrow-left:before { + content: "\f060"; } + +.fa-arrow-right:before { + content: "\f061"; } + +.fa-arrow-up:before { + content: "\f062"; } + +.fa-arrow-down:before { + content: "\f063"; } + +.fa-mail-forward:before, +.fa-share:before { + content: "\f064"; } + +.fa-expand:before { + content: "\f065"; } + +.fa-compress:before { + content: "\f066"; } + +.fa-plus:before { + content: "\f067"; } + +.fa-minus:before { + content: "\f068"; } + +.fa-asterisk:before { + content: "\f069"; } + +.fa-exclamation-circle:before { + content: "\f06a"; } + +.fa-gift:before { + content: "\f06b"; } + +.fa-leaf:before { + content: "\f06c"; } + +.fa-fire:before { + content: "\f06d"; } + +.fa-eye:before { + content: "\f06e"; } + +.fa-eye-slash:before { + content: "\f070"; } + +.fa-warning:before, +.fa-exclamation-triangle:before { + content: "\f071"; } + +.fa-plane:before { + content: "\f072"; } + +.fa-calendar:before { + content: "\f073"; } + +.fa-random:before { + content: "\f074"; } + +.fa-comment:before { + content: "\f075"; } + +.fa-magnet:before { + content: "\f076"; } + +.fa-chevron-up:before { + content: "\f077"; } + +.fa-chevron-down:before { + content: "\f078"; } + +.fa-retweet:before { + content: "\f079"; } + +.fa-shopping-cart:before { + content: "\f07a"; } + +.fa-folder:before { + content: "\f07b"; } + +.fa-folder-open:before { + content: "\f07c"; } + +.fa-arrows-v:before { + content: "\f07d"; } + +.fa-arrows-h:before { + content: "\f07e"; } + +.fa-bar-chart-o:before, +.fa-bar-chart:before { + content: "\f080"; } + +.fa-twitter-square:before { + content: "\f081"; } + +.fa-facebook-square:before { + content: "\f082"; } + +.fa-camera-retro:before { + content: "\f083"; } + +.fa-key:before { + content: "\f084"; } + +.fa-gears:before, +.fa-cogs:before { + content: "\f085"; } + +.fa-comments:before { + content: "\f086"; } + +.fa-thumbs-o-up:before { + content: "\f087"; } + +.fa-thumbs-o-down:before { + content: "\f088"; } + +.fa-star-half:before { + content: "\f089"; } + +.fa-heart-o:before { + content: "\f08a"; } + +.fa-sign-out:before { + content: "\f08b"; } + +.fa-linkedin-square:before { + content: "\f08c"; } + +.fa-thumb-tack:before { + content: "\f08d"; } + +.fa-external-link:before { + content: "\f08e"; } + +.fa-sign-in:before { + content: "\f090"; } + +.fa-trophy:before { + content: "\f091"; } + +.fa-github-square:before { + content: "\f092"; } + +.fa-upload:before { + content: "\f093"; } + +.fa-lemon-o:before { + content: "\f094"; } + +.fa-phone:before { + content: "\f095"; } + +.fa-square-o:before { + content: "\f096"; } + +.fa-bookmark-o:before { + content: "\f097"; } + +.fa-phone-square:before { + content: "\f098"; } + +.fa-twitter:before { + content: "\f099"; } + +.fa-facebook-f:before, +.fa-facebook:before { + content: "\f09a"; } + +.fa-github:before { + content: "\f09b"; } + +.fa-unlock:before { + content: "\f09c"; } + +.fa-credit-card:before { + content: "\f09d"; } + +.fa-feed:before, +.fa-rss:before { + content: "\f09e"; } + +.fa-hdd-o:before { + content: "\f0a0"; } + +.fa-bullhorn:before { + content: "\f0a1"; } + +.fa-bell-o:before { + content: "\f0f3"; } + +.fa-certificate:before { + content: "\f0a3"; } + +.fa-hand-o-right:before { + content: "\f0a4"; } + +.fa-hand-o-left:before { + content: "\f0a5"; } + +.fa-hand-o-up:before { + content: "\f0a6"; } + +.fa-hand-o-down:before { + content: "\f0a7"; } + +.fa-arrow-circle-left:before { + content: "\f0a8"; } + +.fa-arrow-circle-right:before { + content: "\f0a9"; } + +.fa-arrow-circle-up:before { + content: "\f0aa"; } + +.fa-arrow-circle-down:before { + content: "\f0ab"; } + +.fa-globe:before { + content: "\f0ac"; } + +.fa-globe-e:before { + content: "\f304"; } + +.fa-globe-w:before { + content: "\f305"; } + +.fa-wrench:before { + content: "\f0ad"; } + +.fa-tasks:before { + content: "\f0ae"; } + +.fa-filter:before { + content: "\f0b0"; } + +.fa-briefcase:before { + content: "\f0b1"; } + +.fa-arrows-alt:before { + content: "\f0b2"; } + +.fa-community:before, +.fa-group:before, +.fa-users:before { + content: "\f0c0"; } + +.fa-chain:before, +.fa-link:before { + content: "\f0c1"; } + +.fa-cloud:before { + content: "\f0c2"; } + +.fa-flask:before { + content: "\f0c3"; } + +.fa-cut:before, +.fa-scissors:before { + content: "\f0c4"; } + +.fa-copy:before, +.fa-files-o:before { + content: "\f0c5"; } + +.fa-paperclip:before { + content: "\f0c6"; } + +.fa-save:before, +.fa-floppy-o:before { + content: "\f0c7"; } + +.fa-square:before { + content: "\f0c8"; } + +.fa-navicon:before, +.fa-reorder:before, +.fa-bars:before { + content: "\f0c9"; } + +.fa-list-ul:before { + content: "\f0ca"; } + +.fa-list-ol:before { + content: "\f0cb"; } + +.fa-strikethrough:before { + content: "\f0cc"; } + +.fa-underline:before { + content: "\f0cd"; } + +.fa-table:before { + content: "\f0ce"; } + +.fa-magic:before { + content: "\f0d0"; } + +.fa-truck:before { + content: "\f0d1"; } + +.fa-pinterest:before { + content: "\f0d2"; } + +.fa-pinterest-square:before { + content: "\f0d3"; } + +.fa-google-plus-square:before { + content: "\f0d4"; } + +.fa-google-plus-g:before, +.fa-google-plus:before { + content: "\f0d5"; } + +.fa-money:before { + content: "\f0d6"; } + +.fa-caret-down:before { + content: "\f0d7"; } + +.fa-caret-up:before { + content: "\f0d8"; } + +.fa-caret-left:before { + content: "\f0d9"; } + +.fa-caret-right:before { + content: "\f0da"; } + +.fa-columns:before { + content: "\f0db"; } + +.fa-unsorted:before, +.fa-sort:before { + content: "\f0dc"; } + +.fa-sort-down:before, +.fa-sort-desc:before { + content: "\f0dd"; } + +.fa-sort-up:before, +.fa-sort-asc:before { + content: "\f0de"; } + +.fa-envelope:before { + content: "\f0e0"; } + +.fa-linkedin:before { + content: "\f0e1"; } + +.fa-rotate-left:before, +.fa-undo:before { + content: "\f0e2"; } + +.fa-legal:before, +.fa-gavel:before { + content: "\f0e3"; } + +.fa-dashboard:before, +.fa-tachometer:before { + content: "\f0e4"; } + +.fa-comment-o:before { + content: "\f0e5"; } + +.fa-comments-o:before { + content: "\f0e6"; } + +.fa-flash:before, +.fa-bolt:before { + content: "\f0e7"; } + +.fa-sitemap:before { + content: "\f0e8"; } + +.fa-umbrella:before { + content: "\f0e9"; } + +.fa-paste:before, +.fa-clipboard:before { + content: "\f0ea"; } + +.fa-lightbulb-o:before { + content: "\f0eb"; } + +.fa-exchange:before { + content: "\f0ec"; } + +.fa-cloud-download:before { + content: "\f0ed"; } + +.fa-cloud-upload:before { + content: "\f0ee"; } + +.fa-user-md:before { + content: "\f0f0"; } + +.fa-stethoscope:before { + content: "\f0f1"; } + +.fa-suitcase:before { + content: "\f0f2"; } + +.fa-bell:before { + content: "\f0a2"; } + +.fa-coffee:before { + content: "\f0f4"; } + +.fa-utensils:before, +.fa-cutlery:before { + content: "\f0f5"; } + +.fa-file-text-o:before { + content: "\f0f6"; } + +.fa-building-o:before { + content: "\f0f7"; } + +.fa-hospital-o:before { + content: "\f0f8"; } + +.fa-ambulance:before { + content: "\f0f9"; } + +.fa-medkit:before { + content: "\f0fa"; } + +.fa-fighter-jet:before { + content: "\f0fb"; } + +.fa-beer:before { + content: "\f0fc"; } + +.fa-h-square:before { + content: "\f0fd"; } + +.fa-plus-square:before { + content: "\f0fe"; } + +.fa-angle-double-left:before { + content: "\f100"; } + +.fa-angle-double-right:before { + content: "\f101"; } + +.fa-angle-double-up:before { + content: "\f102"; } + +.fa-angle-double-down:before { + content: "\f103"; } + +.fa-angle-left:before { + content: "\f104"; } + +.fa-angle-right:before { + content: "\f105"; } + +.fa-angle-up:before { + content: "\f106"; } + +.fa-angle-down:before { + content: "\f107"; } + +.fa-desktop:before { + content: "\f108"; } + +.fa-laptop:before { + content: "\f109"; } + +.fa-tablet:before { + content: "\f10a"; } + +.fa-mobile-phone:before, +.fa-mobile:before { + content: "\f10b"; } + +.fa-circle-o:before { + content: "\f10c"; } + +.fa-quote-left:before { + content: "\f10d"; } + +.fa-quote-right:before { + content: "\f10e"; } + +.fa-spinner:before { + content: "\f110"; } + +.fa-circle:before { + content: "\f111"; } + +.fa-mail-reply:before, +.fa-reply:before { + content: "\f112"; } + +.fa-github-alt:before { + content: "\f113"; } + +.fa-folder-o:before { + content: "\f114"; } + +.fa-folder-open-o:before { + content: "\f115"; } + +.fa-smile-o:before { + content: "\f118"; } + +.fa-frown-o:before { + content: "\f119"; } + +.fa-meh-o:before { + content: "\f11a"; } + +.fa-gamepad:before { + content: "\f11b"; } + +.fa-keyboard-o:before { + content: "\f11c"; } + +.fa-flag-o:before { + content: "\f11d"; } + +.fa-flag-checkered:before { + content: "\f11e"; } + +.fa-terminal:before { + content: "\f120"; } + +.fa-code:before { + content: "\f121"; } + +.fa-mail-reply-all:before, +.fa-reply-all:before { + content: "\f122"; } + +.fa-star-half-empty:before, +.fa-star-half-full:before, +.fa-star-half-o:before { + content: "\f123"; } + +.fa-location-arrow:before { + content: "\f124"; } + +.fa-crop:before { + content: "\f125"; } + +.fa-code-fork:before { + content: "\f126"; } + +.fa-unlink:before, +.fa-chain-broken:before { + content: "\f127"; } + +.fa-question:before { + content: "\f128"; } + +.fa-info:before { + content: "\f129"; } + +.fa-exclamation:before { + content: "\f12a"; } + +.fa-superscript:before { + content: "\f12b"; } + +.fa-subscript:before { + content: "\f12c"; } + +.fa-eraser:before { + content: "\f12d"; } + +.fa-puzzle-piece:before { + content: "\f12e"; } + +.fa-microphone:before { + content: "\f130"; } + +.fa-microphone-slash:before { + content: "\f131"; } + +.fa-shield:before { + content: "\f132"; } + +.fa-calendar-o:before { + content: "\f133"; } + +.fa-fire-extinguisher:before { + content: "\f134"; } + +.fa-rocket:before { + content: "\f135"; } + +.fa-maxcdn:before { + content: "\f136"; } + +.fa-chevron-circle-left:before { + content: "\f137"; } + +.fa-chevron-circle-right:before { + content: "\f138"; } + +.fa-chevron-circle-up:before { + content: "\f139"; } + +.fa-chevron-circle-down:before { + content: "\f13a"; } + +.fa-html5:before { + content: "\f13b"; } + +.fa-css3:before { + content: "\f13c"; } + +.fa-anchor:before { + content: "\f13d"; } + +.fa-unlock-alt:before { + content: "\f13e"; } + +.fa-bullseye:before { + content: "\f140"; } + +.fa-ellipsis-h:before { + content: "\f141"; } + +.fa-ellipsis-v:before { + content: "\f142"; } + +.fa-rss-square:before { + content: "\f143"; } + +.fa-play-circle:before { + content: "\f144"; } + +.fa-ticket:before { + content: "\f145"; } + +.fa-minus-square:before { + content: "\f146"; } + +.fa-minus-square-o:before { + content: "\f147"; } + +.fa-level-up:before { + content: "\f148"; } + +.fa-level-down:before { + content: "\f149"; } + +.fa-check-square:before { + content: "\f14a"; } + +.fa-pencil-square:before { + content: "\f14b"; } + +.fa-external-link-square:before { + content: "\f14c"; } + +.fa-share-square:before { + content: "\f14d"; } + +.fa-compass:before { + content: "\f14e"; } + +.fa-toggle-down:before, +.fa-caret-square-o-down:before { + content: "\f150"; } + +.fa-toggle-up:before, +.fa-caret-square-o-up:before { + content: "\f151"; } + +.fa-toggle-right:before, +.fa-caret-square-o-right:before { + content: "\f152"; } + +.fa-euro:before, +.fa-eur:before { + content: "\f153"; } + +.fa-pound:before, +.fa-gbp:before { + content: "\f154"; } + +.fa-dollar:before, +.fa-usd:before { + content: "\f155"; } + +.fa-rupee:before, +.fa-inr:before { + content: "\f156"; } + +.fa-cny:before, +.fa-rmb:before, +.fa-yen:before, +.fa-jpy:before { + content: "\f157"; } + +.fa-ruble:before, +.fa-rouble:before, +.fa-rub:before { + content: "\f158"; } + +.fa-won:before, +.fa-krw:before { + content: "\f159"; } + +.fa-bitcoin:before, +.fa-btc:before { + content: "\f15a"; } + +.fa-file:before { + content: "\f15b"; } + +.fa-file-text:before { + content: "\f15c"; } + +.fa-sort-alpha-down:before, +.fa-sort-alpha-asc:before { + content: "\f15d"; } + +.fa-sort-alpha-up:before, +.fa-sort-alpha-desc:before { + content: "\f15e"; } + +.fa-sort-amount-down:before, +.fa-sort-amount-asc:before { + content: "\f160"; } + +.fa-sort-amount-up:before, +.fa-sort-amount-desc:before { + content: "\f161"; } + +.fa-sort-numeric-down:before, +.fa-sort-numeric-asc:before { + content: "\f162"; } + +.fa-sort-numeric-up:before, +.fa-sort-numeric-desc:before { + content: "\f163"; } + +.fa-thumbs-up:before { + content: "\f164"; } + +.fa-thumbs-down:before { + content: "\f165"; } + +.fa-youtube-square:before { + content: "\f166"; } + +.fa-youtube:before { + content: "\f167"; } + +.fa-xing:before { + content: "\f168"; } + +.fa-xing-square:before { + content: "\f169"; } + +.fa-youtube-play:before { + content: "\f16a"; } + +.fa-dropbox:before { + content: "\f16b"; } + +.fa-stack-overflow:before { + content: "\f16c"; } + +.fa-instagram:before { + content: "\f16d"; } + +.fa-flickr:before { + content: "\f16e"; } + +.fa-adn:before { + content: "\f170"; } + +.fa-bitbucket:before { + content: "\f171"; } + +.fa-bitbucket-square:before { + content: "\f172"; } + +.fa-tumblr:before { + content: "\f173"; } + +.fa-tumblr-square:before { + content: "\f174"; } + +.fa-long-arrow-down:before { + content: "\f175"; } + +.fa-long-arrow-up:before { + content: "\f176"; } + +.fa-long-arrow-left:before { + content: "\f177"; } + +.fa-long-arrow-right:before { + content: "\f178"; } + +.fa-apple:before { + content: "\f179"; } + +.fa-windows:before { + content: "\f17a"; } + +.fa-android:before { + content: "\f17b"; } + +.fa-linux:before { + content: "\f17c"; } + +.fa-dribbble:before { + content: "\f17d"; } + +.fa-skype:before { + content: "\f17e"; } + +.fa-foursquare:before { + content: "\f180"; } + +.fa-trello:before { + content: "\f181"; } + +.fa-female:before { + content: "\f182"; } + +.fa-male:before { + content: "\f183"; } + +.fa-gittip:before, +.fa-gratipay:before { + content: "\f184"; } + +.fa-sun-o:before { + content: "\f185"; } + +.fa-moon-o:before { + content: "\f186"; } + +.fa-archive:before { + content: "\f187"; } + +.fa-bug:before { + content: "\f188"; } + +.fa-vk:before { + content: "\f189"; } + +.fa-weibo:before { + content: "\f18a"; } + +.fa-renren:before { + content: "\f18b"; } + +.fa-pagelines:before { + content: "\f18c"; } + +.fa-stack-exchange:before { + content: "\f18d"; } + +.fa-arrow-circle-o-right:before { + content: "\f18e"; } + +.fa-arrow-circle-o-left:before { + content: "\f190"; } + +.fa-toggle-left:before, +.fa-caret-square-o-left:before { + content: "\f191"; } + +.fa-dot-circle-o:before { + content: "\f192"; } + +.fa-wheelchair:before { + content: "\f193"; } + +.fa-vimeo-square:before { + content: "\f194"; } + +.fa-turkish-lira:before, +.fa-try:before { + content: "\f195"; } + +.fa-plus-square-o:before { + content: "\f196"; } + +.fa-space-shuttle:before { + content: "\f197"; } + +.fa-slack:before { + content: "\f198"; } + +.fa-envelope-square:before { + content: "\f199"; } + +.fa-wordpress:before { + content: "\f19a"; } + +.fa-openid:before { + content: "\f19b"; } + +.fa-institution:before, +.fa-bank:before, +.fa-university:before { + content: "\f19c"; } + +.fa-mortar-board:before, +.fa-graduation-cap:before { + content: "\f19d"; } + +.fa-yahoo:before { + content: "\f19e"; } + +.fa-google:before { + content: "\f1a0"; } + +.fa-reddit:before { + content: "\f1a1"; } + +.fa-reddit-square:before { + content: "\f1a2"; } + +.fa-stumbleupon-circle:before { + content: "\f1a3"; } + +.fa-stumbleupon:before { + content: "\f1a4"; } + +.fa-delicious:before { + content: "\f1a5"; } + +.fa-digg:before { + content: "\f1a6"; } + +.fa-drupal:before { + content: "\f1a9"; } + +.fa-joomla:before { + content: "\f1aa"; } + +.fa-language:before { + content: "\f1ab"; } + +.fa-fax:before { + content: "\f1ac"; } + +.fa-building:before { + content: "\f1ad"; } + +.fa-child:before { + content: "\f1ae"; } + +.fa-paw:before { + content: "\f1b0"; } + +.fa-utensil-spoon:before, +.fa-spoon:before { + content: "\f1b1"; } + +.fa-cube:before { + content: "\f1b2"; } + +.fa-cubes:before { + content: "\f1b3"; } + +.fa-behance:before { + content: "\f1b4"; } + +.fa-behance-square:before { + content: "\f1b5"; } + +.fa-steam:before { + content: "\f1b6"; } + +.fa-steam-square:before { + content: "\f1b7"; } + +.fa-recycle:before { + content: "\f1b8"; } + +.fa-automobile:before, +.fa-car:before { + content: "\f1b9"; } + +.fa-cab:before, +.fa-taxi:before { + content: "\f1ba"; } + +.fa-tree:before { + content: "\f1bb"; } + +.fa-spotify:before { + content: "\f1bc"; } + +.fa-deviantart:before { + content: "\f1bd"; } + +.fa-soundcloud:before { + content: "\f1be"; } + +.fa-database:before { + content: "\f1c0"; } + +.fa-file-pdf-o:before { + content: "\f1c1"; } + +.fa-file-word-o:before { + content: "\f1c2"; } + +.fa-file-excel-o:before { + content: "\f1c3"; } + +.fa-file-powerpoint-o:before { + content: "\f1c4"; } + +.fa-file-photo-o:before, +.fa-file-picture-o:before, +.fa-file-image-o:before { + content: "\f1c5"; } + +.fa-file-zip-o:before, +.fa-file-archive-o:before { + content: "\f1c6"; } + +.fa-file-sound-o:before, +.fa-file-audio-o:before { + content: "\f1c7"; } + +.fa-file-movie-o:before, +.fa-file-video-o:before { + content: "\f1c8"; } + +.fa-file-code-o:before { + content: "\f1c9"; } + +.fa-vine:before { + content: "\f1ca"; } + +.fa-codepen:before { + content: "\f1cb"; } + +.fa-jsfiddle:before { + content: "\f1cc"; } + +.fa-life-bouy:before, +.fa-life-buoy:before, +.fa-life-saver:before, +.fa-support:before, +.fa-life-ring:before { + content: "\f1cd"; } + +.fa-circle-o-notch:before { + content: "\f1ce"; } + +.fa-ra:before, +.fa-resistance:before, +.fa-rebel:before { + content: "\f1d0"; } + +.fa-ge:before, +.fa-empire:before { + content: "\f1d1"; } + +.fa-git-square:before { + content: "\f1d2"; } + +.fa-git:before { + content: "\f1d3"; } + +.fa-y-combinator-square:before, +.fa-yc-square:before, +.fa-hacker-news:before { + content: "\f1d4"; } + +.fa-tencent-weibo:before { + content: "\f1d5"; } + +.fa-qq:before { + content: "\f1d6"; } + +.fa-wechat:before, +.fa-weixin:before { + content: "\f1d7"; } + +.fa-send:before, +.fa-paper-plane:before { + content: "\f1d8"; } + +.fa-send-o:before, +.fa-paper-plane-o:before { + content: "\f1d9"; } + +.fa-history:before { + content: "\f1da"; } + +.fa-circle-thin:before { + content: "\f1db"; } + +.fa-heading:before, +.fa-header:before { + content: "\f1dc"; } + +.fa-paragraph:before { + content: "\f1dd"; } + +.fa-sliders:before { + content: "\f1de"; } + +.fa-share-alt:before { + content: "\f1e0"; } + +.fa-share-alt-square:before { + content: "\f1e1"; } + +.fa-bomb:before { + content: "\f1e2"; } + +.fa-soccer-ball-o:before, +.fa-futbol-o:before { + content: "\f1e3"; } + +.fa-tty:before { + content: "\f1e4"; } + +.fa-binoculars:before { + content: "\f1e5"; } + +.fa-plug:before { + content: "\f1e6"; } + +.fa-slideshare:before { + content: "\f1e7"; } + +.fa-twitch:before { + content: "\f1e8"; } + +.fa-yelp:before { + content: "\f1e9"; } + +.fa-newspaper-o:before { + content: "\f1ea"; } + +.fa-wifi:before { + content: "\f1eb"; } + +.fa-calculator:before { + content: "\f1ec"; } + +.fa-paypal:before { + content: "\f1ed"; } + +.fa-google-wallet:before { + content: "\f1ee"; } + +.fa-cc-visa:before { + content: "\f1f0"; } + +.fa-cc-mastercard:before { + content: "\f1f1"; } + +.fa-cc-discover:before { + content: "\f1f2"; } + +.fa-cc-amex:before { + content: "\f1f3"; } + +.fa-cc-paypal:before { + content: "\f1f4"; } + +.fa-cc-stripe:before { + content: "\f1f5"; } + +.fa-bell-slash:before { + content: "\f1f6"; } + +.fa-bell-slash-o:before { + content: "\f1f7"; } + +.fa-trash:before { + content: "\f1f8"; } + +.fa-copyright:before { + content: "\f1f9"; } + +.fa-at:before { + content: "\f1fa"; } + +.fa-eyedropper:before { + content: "\f1fb"; } + +.fa-paint-brush:before { + content: "\f1fc"; } + +.fa-birthday-cake:before { + content: "\f1fd"; } + +.fa-area-chart:before { + content: "\f1fe"; } + +.fa-pie-chart:before { + content: "\f200"; } + +.fa-line-chart:before { + content: "\f201"; } + +.fa-lastfm:before { + content: "\f202"; } + +.fa-lastfm-square:before { + content: "\f203"; } + +.fa-toggle-off:before { + content: "\f204"; } + +.fa-toggle-on:before { + content: "\f205"; } + +.fa-bicycle:before { + content: "\f206"; } + +.fa-bus:before { + content: "\f207"; } + +.fa-ioxhost:before { + content: "\f208"; } + +.fa-angellist:before { + content: "\f209"; } + +.fa-closed-captioning:before, +.fa-cc:before { + content: "\f20a"; } + +.fa-shekel:before, +.fa-sheqel:before, +.fa-ils:before { + content: "\f20b"; } + +.fa-meanpath:before { + content: "\f20c"; } + +.fa-buysellads:before { + content: "\f20d"; } + +.fa-connectdevelop:before { + content: "\f20e"; } + +.fa-dashcube:before { + content: "\f210"; } + +.fa-forumbee:before { + content: "\f211"; } + +.fa-leanpub:before { + content: "\f212"; } + +.fa-sellsy:before { + content: "\f213"; } + +.fa-shirtsinbulk:before { + content: "\f214"; } + +.fa-simplybuilt:before { + content: "\f215"; } + +.fa-skyatlas:before { + content: "\f216"; } + +.fa-cart-plus:before { + content: "\f217"; } + +.fa-cart-arrow-down:before { + content: "\f218"; } + +.fa-gem:before, +.fa-diamond:before { + content: "\f219"; } + +.fa-ship:before { + content: "\f21a"; } + +.fa-user-secret:before { + content: "\f21b"; } + +.fa-motorcycle:before { + content: "\f21c"; } + +.fa-street-view:before { + content: "\f21d"; } + +.fa-heartbeat:before { + content: "\f21e"; } + +.fa-venus:before { + content: "\f221"; } + +.fa-mars:before { + content: "\f222"; } + +.fa-mercury:before { + content: "\f223"; } + +.fa-intersex:before, +.fa-transgender:before { + content: "\f224"; } + +.fa-transgender-alt:before { + content: "\f225"; } + +.fa-venus-double:before { + content: "\f226"; } + +.fa-mars-double:before { + content: "\f227"; } + +.fa-venus-mars:before { + content: "\f228"; } + +.fa-mars-stroke:before { + content: "\f229"; } + +.fa-mars-stroke-v:before { + content: "\f22a"; } + +.fa-mars-stroke-h:before { + content: "\f22b"; } + +.fa-neuter:before { + content: "\f22c"; } + +.fa-genderless:before { + content: "\f22d"; } + +.fa-facebook-official:before { + content: "\f230"; } + +.fa-pinterest-p:before { + content: "\f231"; } + +.fa-whatsapp:before { + content: "\f232"; } + +.fa-server:before { + content: "\f233"; } + +.fa-user-plus:before { + content: "\f234"; } + +.fa-user-times:before { + content: "\f235"; } + +.fa-hotel:before, +.fa-bed:before { + content: "\f236"; } + +.fa-viacoin:before { + content: "\f237"; } + +.fa-train:before { + content: "\f238"; } + +.fa-subway:before { + content: "\f239"; } + +.fa-medium:before { + content: "\f23a"; } + +.fa-medium-square:before { + content: "\f2f8"; } + +.fa-yc:before, +.fa-y-combinator:before { + content: "\f23b"; } + +.fa-optin-monster:before { + content: "\f23c"; } + +.fa-opencart:before { + content: "\f23d"; } + +.fa-expeditedssl:before { + content: "\f23e"; } + +.fa-battery-4:before, +.fa-battery:before, +.fa-battery-full:before { + content: "\f240"; } + +.fa-battery-3:before, +.fa-battery-three-quarters:before { + content: "\f241"; } + +.fa-battery-2:before, +.fa-battery-half:before { + content: "\f242"; } + +.fa-battery-1:before, +.fa-battery-quarter:before { + content: "\f243"; } + +.fa-battery-0:before, +.fa-battery-empty:before { + content: "\f244"; } + +.fa-mouse-pointer:before { + content: "\f245"; } + +.fa-i-cursor:before { + content: "\f246"; } + +.fa-object-group:before { + content: "\f247"; } + +.fa-object-ungroup:before { + content: "\f248"; } + +.fa-sticky-note:before { + content: "\f249"; } + +.fa-sticky-note-o:before { + content: "\f24a"; } + +.fa-cc-jcb:before { + content: "\f24b"; } + +.fa-cc-diners-club:before { + content: "\f24c"; } + +.fa-clone:before { + content: "\f24d"; } + +.fa-balance-scale:before { + content: "\f24e"; } + +.fa-hourglass-o:before { + content: "\f250"; } + +.fa-hourglass-1:before, +.fa-hourglass-start:before { + content: "\f251"; } + +.fa-hourglass-2:before, +.fa-hourglass-half:before { + content: "\f252"; } + +.fa-hourglass-3:before, +.fa-hourglass-end:before { + content: "\f253"; } + +.fa-hourglass:before { + content: "\f254"; } + +.fa-hand-grab-o:before, +.fa-hand-rock-o:before { + content: "\f255"; } + +.fa-hand-stop-o:before, +.fa-hand-paper-o:before { + content: "\f256"; } + +.fa-hand-scissors-o:before { + content: "\f257"; } + +.fa-hand-lizard-o:before { + content: "\f258"; } + +.fa-hand-spock-o:before { + content: "\f259"; } + +.fa-hand-pointer-o:before { + content: "\f25a"; } + +.fa-hand-peace-o:before { + content: "\f25b"; } + +.fa-trademark:before { + content: "\f25c"; } + +.fa-registered:before { + content: "\f25d"; } + +.fa-creative-commons:before { + content: "\f25e"; } + +.fa-gg:before { + content: "\f260"; } + +.fa-gg-circle:before { + content: "\f261"; } + +.fa-tripadvisor:before { + content: "\f262"; } + +.fa-odnoklassniki:before { + content: "\f263"; } + +.fa-odnoklassniki-square:before { + content: "\f264"; } + +.fa-get-pocket:before { + content: "\f265"; } + +.fa-wikipedia-w:before { + content: "\f266"; } + +.fa-safari:before { + content: "\f267"; } + +.fa-chrome:before { + content: "\f268"; } + +.fa-firefox:before { + content: "\f269"; } + +.fa-opera:before { + content: "\f26a"; } + +.fa-internet-explorer:before { + content: "\f26b"; } + +.fa-tv:before, +.fa-television:before { + content: "\f26c"; } + +.fa-contao:before { + content: "\f26d"; } + +.fa-500px:before { + content: "\f26e"; } + +.fa-amazon:before { + content: "\f270"; } + +.fa-calendar-plus-o:before { + content: "\f271"; } + +.fa-calendar-minus-o:before { + content: "\f272"; } + +.fa-calendar-times-o:before { + content: "\f273"; } + +.fa-calendar-check-o:before { + content: "\f274"; } + +.fa-industry:before { + content: "\f275"; } + +.fa-map-pin:before { + content: "\f276"; } + +.fa-map-signs:before { + content: "\f277"; } + +.fa-map-o:before { + content: "\f278"; } + +.fa-map:before { + content: "\f279"; } + +.fa-commenting:before { + content: "\f27a"; } + +.fa-commenting-o:before { + content: "\f27b"; } + +.fa-houzz:before { + content: "\f27c"; } + +.fa-vimeo-v:before, +.fa-vimeo:before { + content: "\f27d"; } + +.fa-black-tie:before { + content: "\f27e"; } + +.fa-fonticons:before { + content: "\f280"; } + +.fa-reddit-alien:before { + content: "\f281"; } + +.fa-edge:before { + content: "\f282"; } + +.fa-credit-card-alt:before { + content: "\f283"; } + +.fa-codiepie:before { + content: "\f284"; } + +.fa-modx:before { + content: "\f285"; } + +.fa-fort-awesome:before { + content: "\f286"; } + +.fa-usb:before { + content: "\f287"; } + +.fa-product-hunt:before { + content: "\f288"; } + +.fa-mixcloud:before { + content: "\f289"; } + +.fa-scribd:before { + content: "\f28a"; } + +.fa-pause-circle:before { + content: "\f28b"; } + +.fa-pause-circle-o:before { + content: "\f28c"; } + +.fa-stop-circle:before { + content: "\f28d"; } + +.fa-stop-circle-o:before { + content: "\f28e"; } + +.fa-shopping-bag:before { + content: "\f290"; } + +.fa-shopping-basket:before { + content: "\f291"; } + +.fa-hashtag:before { + content: "\f292"; } + +.fa-bluetooth:before { + content: "\f293"; } + +.fa-bluetooth-b:before { + content: "\f294"; } + +.fa-percent:before { + content: "\f295"; } + +.fa-gitlab:before { + content: "\f296"; } + +.fa-wpbeginner:before { + content: "\f297"; } + +.fa-wpforms:before { + content: "\f298"; } + +.fa-envira:before { + content: "\f299"; } + +.fa-universal-access:before { + content: "\f29a"; } + +.fa-wheelchair-alt:before { + content: "\f29b"; } + +.fa-question-circle-o:before { + content: "\f29c"; } + +.fa-blind:before { + content: "\f29d"; } + +.fa-audio-description:before { + content: "\f29e"; } + +.fa-phone-volume:before, +.fa-volume-control-phone:before { + content: "\f2a0"; } + +.fa-braille:before { + content: "\f2a1"; } + +.fa-assistive-listening-systems:before { + content: "\f2a2"; } + +.fa-asl-interpreting:before, +.fa-american-sign-language-interpreting:before { + content: "\f2a3"; } + +.fa-deafness:before, +.fa-hard-of-hearing:before, +.fa-deaf:before { + content: "\f2a4"; } + +.fa-glide:before { + content: "\f2a5"; } + +.fa-glide-g:before { + content: "\f2a6"; } + +.fa-signing:before, +.fa-sign-language:before { + content: "\f2a7"; } + +.fa-low-vision:before { + content: "\f2a8"; } + +.fa-viadeo:before { + content: "\f2a9"; } + +.fa-viadeo-square:before { + content: "\f2aa"; } + +.fa-snapchat:before { + content: "\f2ab"; } + +.fa-snapchat-ghost:before { + content: "\f2ac"; } + +.fa-snapchat-square:before { + content: "\f2ad"; } + +.fa-first-order:before { + content: "\f2b0"; } + +.fa-yoast:before { + content: "\f2b1"; } + +.fa-themeisle:before { + content: "\f2b2"; } + +.fa-google-plus-circle:before, +.fa-google-plus-official:before { + content: "\f2b3"; } + +.fa-fa:before, +.fa-font-awesome:before { + content: "\f2b4"; } + +.fa-handshake-o:before { + content: "\f2b5"; } + +.fa-envelope-open:before { + content: "\f2b6"; } + +.fa-envelope-open-o:before { + content: "\f2b7"; } + +.fa-linode:before { + content: "\f2b8"; } + +.fa-address-book:before { + content: "\f2b9"; } + +.fa-address-book-o:before { + content: "\f2ba"; } + +.fa-vcard:before, +.fa-address-card:before { + content: "\f2bb"; } + +.fa-vcard-o:before, +.fa-address-card-o:before { + content: "\f2bc"; } + +.fa-user-circle:before { + content: "\f2bd"; } + +.fa-user-circle-o:before { + content: "\f2be"; } + +.fa-user-o:before { + content: "\f2c0"; } + +.fa-id-badge:before { + content: "\f2c1"; } + +.fa-drivers-license:before, +.fa-id-card:before { + content: "\f2c2"; } + +.fa-drivers-license-o:before, +.fa-id-card-o:before { + content: "\f2c3"; } + +.fa-quora:before { + content: "\f2c4"; } + +.fa-free-code-camp:before { + content: "\f2c5"; } + +.fa-telegram:before { + content: "\f2c6"; } + +.fa-thermometer-4:before, +.fa-thermometer:before, +.fa-thermometer-full:before { + content: "\f2c7"; } + +.fa-thermometer-3:before, +.fa-thermometer-three-quarters:before { + content: "\f2c8"; } + +.fa-thermometer-2:before, +.fa-thermometer-half:before { + content: "\f2c9"; } + +.fa-thermometer-1:before, +.fa-thermometer-quarter:before { + content: "\f2ca"; } + +.fa-thermometer-0:before, +.fa-thermometer-empty:before { + content: "\f2cb"; } + +.fa-shower:before { + content: "\f2cc"; } + +.fa-bathtub:before, +.fa-s15:before, +.fa-bath:before { + content: "\f2cd"; } + +.fa-podcast:before { + content: "\f2ce"; } + +.fa-window-maximize:before { + content: "\f2d0"; } + +.fa-window-minimize:before { + content: "\f2d1"; } + +.fa-window-restore:before { + content: "\f2d2"; } + +.fa-times-rectangle:before, +.fa-window-close:before { + content: "\f2d3"; } + +.fa-times-rectangle-o:before, +.fa-window-close-o:before { + content: "\f2d4"; } + +.fa-bandcamp:before { + content: "\f2d5"; } + +.fa-grav:before { + content: "\f2d6"; } + +.fa-etsy:before { + content: "\f2d7"; } + +.fa-imdb:before { + content: "\f2d8"; } + +.fa-ravelry:before { + content: "\f2d9"; } + +.fa-eercast:before { + content: "\f2da"; } + +.fa-microchip:before { + content: "\f2db"; } + +.fa-snowflake-o:before { + content: "\f2dc"; } + +.fa-superpowers:before { + content: "\f2dd"; } + +.fa-wpexplorer:before { + content: "\f2de"; } + +.fa-meetup:before { + content: "\f2e0"; } + +.fa-mastodon:before { + content: "\f2e1"; } + +.fa-mastodon-alt:before { + content: "\f2e2"; } + +.fa-fork-circle:before, +.fa-fork-awesome:before { + content: "\f2e3"; } + +.fa-peertube:before { + content: "\f2e4"; } + +.fa-diaspora:before { + content: "\f2e5"; } + +.fa-friendica:before { + content: "\f2e6"; } + +.fa-gnu-social:before { + content: "\f2e7"; } + +.fa-liberapay-square:before { + content: "\f2e8"; } + +.fa-liberapay:before { + content: "\f2e9"; } + +.fa-ssb:before, +.fa-scuttlebutt:before { + content: "\f2ea"; } + +.fa-hubzilla:before { + content: "\f2eb"; } + +.fa-social-home:before { + content: "\f2ec"; } + +.fa-artstation:before { + content: "\f2ed"; } + +.fa-discord:before { + content: "\f2ee"; } + +.fa-discord-alt:before { + content: "\f2ef"; } + +.fa-patreon:before { + content: "\f2f0"; } + +.fa-snowdrift:before { + content: "\f2f1"; } + +.fa-activitypub:before { + content: "\f2f2"; } + +.fa-ethereum:before { + content: "\f2f3"; } + +.fa-keybase:before { + content: "\f2f4"; } + +.fa-shaarli:before { + content: "\f2f5"; } + +.fa-shaarli-o:before { + content: "\f2f6"; } + +.fa-cut-key:before, +.fa-key-modern:before { + content: "\f2f7"; } + +.fa-xmpp:before { + content: "\f2f9"; } + +.fa-archive-org:before { + content: "\f2fc"; } + +.fa-freedombox:before { + content: "\f2fd"; } + +.fa-facebook-messenger:before { + content: "\f2fe"; } + +.fa-debian:before { + content: "\f2ff"; } + +.fa-mastodon-square:before { + content: "\f300"; } + +.fa-tipeee:before { + content: "\f301"; } + +.fa-react:before { + content: "\f302"; } + +.fa-dogmazic:before { + content: "\f303"; } + +.fa-zotero:before { + content: "\f309"; } + +.fa-nodejs:before { + content: "\f308"; } + +.fa-nextcloud:before { + content: "\f306"; } + +.fa-nextcloud-square:before { + content: "\f307"; } + +.fa-hackaday:before { + content: "\f30a"; } + +.fa-laravel:before { + content: "\f30b"; } + +.fa-signalapp:before { + content: "\f30c"; } + +.fa-gnupg:before { + content: "\f30d"; } + +.fa-php:before { + content: "\f30e"; } + +.fa-ffmpeg:before { + content: "\f30f"; } + +.fa-joplin:before { + content: "\f310"; } + +.fa-syncthing:before { + content: "\f311"; } + +.fa-inkscape:before { + content: "\f312"; } + +.fa-matrix-org:before { + content: "\f313"; } + +.fa-pixelfed:before { + content: "\f314"; } + +.fa-bootstrap:before { + content: "\f315"; } + +.fa-dev-to:before { + content: "\f316"; } + +.fa-hashnode:before { + content: "\f317"; } + +.fa-jirafeau:before { + content: "\f318"; } + +.fa-emby:before { + content: "\f319"; } + +.fa-wikidata:before { + content: "\f31a"; } + +.fa-gimp:before { + content: "\f31b"; } + +.fa-c:before { + content: "\f31c"; } + +.fa-digitalocean:before { + content: "\f31d"; } + +.fa-att:before { + content: "\f31e"; } + +.fa-gitea:before { + content: "\f31f"; } + +.fa-file-epub:before { + content: "\f321"; } + +.fa-python:before { + content: "\f322"; } + +.fa-archlinux:before { + content: "\f323"; } + +.fa-pleroma:before { + content: "\f324"; } + +.fa-unsplash:before { + content: "\f325"; } + +.fa-hackster:before { + content: "\f326"; } + +.fa-spell-check:before { + content: "\f327"; } + +.fa-moon:before { + content: "\f328"; } + +.fa-sun:before { + content: "\f329"; } + +.fa-f-droid:before { + content: "\f32a"; } + +.fa-biometric:before { + content: "\f32b"; } + +.fa-wire:before { + content: "\f32c"; } + +.fa-tor-onion:before { + content: "\f32e"; } + +.fa-volume-mute:before { + content: "\f32f"; } + +.fa-bell-ringing:before { + content: "\f32d"; } + +.fa-bell-ringing-o:before { + content: "\f330"; } + +.fa-hal:before { + content: "\f333"; } + +.fa-jupyter:before { + content: "\f335"; } + +.fa-julia:before { + content: "\f334"; } + +.fa-classicpress:before { + content: "\f331"; } + +.fa-classicpress-circle:before { + content: "\f332"; } + +.fa-open-collective:before { + content: "\f336"; } + +.fa-orcid:before { + content: "\f337"; } + +.fa-researchgate:before { + content: "\f338"; } + +.fa-funkwhale:before { + content: "\f339"; } + +.fa-askfm:before { + content: "\f33a"; } + +.fa-blockstack:before { + content: "\f33b"; } + +.fa-boardgamegeek:before { + content: "\f33c"; } + +.fa-bunny:before { + content: "\f35f"; } + +.fa-buymeacoffee:before { + content: "\f33d"; } + +.fa-cc-by:before { + content: "\f33e"; } + +.fa-creative-commons-alt:before, +.fa-cc-cc:before { + content: "\f33f"; } + +.fa-cc-nc-eu:before { + content: "\f341"; } + +.fa-cc-nc-jp:before { + content: "\f342"; } + +.fa-cc-nc:before { + content: "\f340"; } + +.fa-cc-nd:before { + content: "\f343"; } + +.fa-cc-pd:before { + content: "\f344"; } + +.fa-cc-remix:before { + content: "\f345"; } + +.fa-cc-sa:before { + content: "\f346"; } + +.fa-cc-share:before { + content: "\f347"; } + +.fa-cc-zero:before { + content: "\f348"; } + +.fa-conway-hacker:before, +.fa-conway-glider:before { + content: "\f349"; } + +.fa-csharp:before { + content: "\f34a"; } + +.fa-email-bulk:before { + content: "\f34b"; } + +.fa-email-bulk-o:before { + content: "\f34c"; } + +.fa-gnu:before { + content: "\f34d"; } + +.fa-google-play:before { + content: "\f34e"; } + +.fa-heroku:before { + content: "\f34f"; } + +.fa-hassio:before, +.fa-home-assistant:before { + content: "\f350"; } + +.fa-java:before { + content: "\f351"; } + +.fa-mariadb:before { + content: "\f352"; } + +.fa-markdown:before { + content: "\f353"; } + +.fa-mysql:before { + content: "\f354"; } + +.fa-nordcast:before { + content: "\f355"; } + +.fa-plume:before { + content: "\f356"; } + +.fa-postgresql:before { + content: "\f357"; } + +.fa-sass-alt:before { + content: "\f359"; } + +.fa-sass:before { + content: "\f358"; } + +.fa-skate:before { + content: "\f35a"; } + +.fa-sketchfab:before { + content: "\f35b"; } + +.fa-tex:before { + content: "\f35c"; } + +.fa-textpattern:before { + content: "\f35d"; } + +.fa-unity:before { + content: "\f35e"; } + +.sr-only { + position: absolute; + width: 1px; + height: 1px; + padding: 0; + margin: -1px; + overflow: hidden; + clip: rect(0, 0, 0, 0); + border: 0; } + +.sr-only-focusable:active, .sr-only-focusable:focus { + position: static; + width: auto; + height: auto; + margin: 0; + overflow: visible; + clip: auto; } + *, *:after, *:before { @@ -9,25 +3041,28 @@ html { body { color: #212121; - background-color: #FAFAFA; - font-family: Merriweather, Georgia, serif; - font-size: 1.6em; - font-weight: 300; + background-color: #fafafa; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", Helvetica, "游ゴシック", "PingFang SC", STXihei, "华文细黑", "Microsoft YaHei", "微软雅黑", SimSun, "宋体", Heiti, "黑体", sans-serif; + font-size: 1.8em; + font-weight: 400; line-height: 1.8em; } @media only screen and (max-width: 768px) { body { font-size: 1.6em; line-height: 1.6em; } } +iframe[src*=disqus] { + color-scheme: light; } + a { - font-weight: 300; + font-weight: 500; color: #1565c0; text-decoration: none; - transition: all .25s ease-in; } + transition: all 0.25s ease-in; } a:focus, a:hover { text-decoration: underline; } p { - margin: 2.0rem 0 2.0rem 0; } + margin: 2rem 0 2rem 0; } h1, h2, @@ -35,17 +3070,44 @@ h3, h4, h5, h6 { - font-family: Lato, Helvetica, sans-serif; - font-weight: 700; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", Helvetica, "游ゴシック", "PingFang SC", STXihei, "华文细黑", "Microsoft YaHei", "微软雅黑", SimSun, "宋体", Heiti, "黑体", sans-serif; + font-weight: 600; color: #000; - margin: 6.4rem 0 3.2rem 0; } + margin: 4rem 0 2.5rem 0; } + h1:hover .heading-link, + h2:hover .heading-link, + h3:hover .heading-link, + h4:hover .heading-link, + h5:hover .heading-link, + h6:hover .heading-link { + visibility: visible; } + h1 .heading-link, + h2 .heading-link, + h3 .heading-link, + h4 .heading-link, + h5 .heading-link, + h6 .heading-link { + color: #1565c0; + font-weight: inherit; + text-decoration: none; + font-size: 80%; + visibility: hidden; } + h1 .title-link, + h2 .title-link, + h3 .title-link, + h4 .title-link, + h5 .title-link, + h6 .title-link { + color: inherit; + font-weight: inherit; + text-decoration: none; } h1 { font-size: 3.2rem; line-height: 3.6rem; } @media only screen and (max-width: 768px) { h1 { - font-size: 3.0rem; + font-size: 3rem; line-height: 3.4rem; } } h2 { font-size: 2.8rem; @@ -53,7 +3115,7 @@ h2 { @media only screen and (max-width: 768px) { h2 { font-size: 2.6rem; - line-height: 3.0rem; } } + line-height: 3rem; } } h3 { font-size: 2.4rem; line-height: 2.8rem; } @@ -66,10 +3128,10 @@ h4 { line-height: 2.6rem; } @media only screen and (max-width: 768px) { h4 { - font-size: 2.0rem; + font-size: 2rem; line-height: 2.4rem; } } h5 { - font-size: 2.0rem; + font-size: 2rem; line-height: 2.4rem; } @media only screen and (max-width: 768px) { h5 { @@ -81,19 +3143,20 @@ h6 { @media only screen and (max-width: 768px) { h6 { font-size: 1.6rem; - line-height: 2.0rem; } } -b, strong { + line-height: 2rem; } } +b, +strong { font-weight: 700; } .highlight > div, .highlight > pre { - margin: 0 0 2rem; + margin: 2rem 0 2rem; padding: 1rem; border-radius: 1rem; } pre { display: block; - font-family: "Source Code Pro", "Lucida Console", monospace; + font-family: SFMono-Regular, Consolas, Liberation Mono, Menlo, monospace; font-size: 1.6rem; font-weight: 400; line-height: 2.6rem; @@ -105,26 +3168,30 @@ pre { color: inherit; } code { - font-family: "Source Code Pro", "Lucida Console", monospace; + font-family: SFMono-Regular, Consolas, Liberation Mono, Menlo, monospace; font-size: 1.6rem; font-weight: 400; - background-color: #E0E0E0; - color: #212121; } + background-color: #e0e0e0; + color: #212121; + border-radius: 0.6rem; + padding: 0.3rem 0.6rem; } blockquote { - border-left: 2px solid #E0E0E0; - padding-left: 2.0rem; + border-left: 2px solid #e0e0e0; + padding-left: 2rem; line-height: 2.2rem; font-weight: 400; font-style: italic; } -th, td { +th, +td { padding: 1.6rem; } table { border-collapse: collapse; } -table td, table th { +table td, +table th { border: 2px solid #000; } table tr:first-child th { @@ -147,6 +3214,16 @@ img { figure { text-align: center; } +.footnotes ol li p { + margin: 0; } + +.preload-transitions * { + -webkit-transition: none !important; + -moz-transition: none !important; + -ms-transition: none !important; + -o-transition: none !important; + transition: none !important; } + .wrapper { display: flex; flex-direction: column; @@ -154,11 +3231,11 @@ figure { width: 100%; } .container { - margin: 0 auto; - max-width: 90.0rem; + margin: 1rem auto; + max-width: 90rem; width: 100%; - padding-left: 2.0rem; - padding-right: 2.0rem; } + padding-left: 2rem; + padding-right: 2rem; } .fab { font-weight: 400; } @@ -178,17 +3255,13 @@ figure { .fas { font-weight: 900; } -img.emoji { - height: 1em; - width: 1em; - margin: 0 .05em 0 .1em; - vertical-align: -0.1em; } - .content { flex: 1; display: flex; margin-top: 1.6rem; margin-bottom: 3.2rem; } + .content article details summary { + cursor: pointer; } .content article header { margin-top: 6.4rem; margin-bottom: 3.2rem; } @@ -198,10 +3271,10 @@ img.emoji { margin: 0; } @media only screen and (max-width: 768px) { .content article header h1 { - font-size: 4.0rem; + font-size: 4rem; line-height: 4.4rem; } } .content article footer { - margin-top: 4.0rem; } + margin-top: 4rem; } .content article footer .see-also { margin: 3.2rem 0; } .content article footer .see-also h3 { @@ -211,7 +3284,7 @@ img.emoji { text-justify: auto; hyphens: auto; } .content .post .post-title { - margin-bottom: .75em; } + margin-bottom: 0.75em; } .content .post .post-meta i { text-align: center; width: 1.6rem; @@ -220,6 +3293,16 @@ img.emoji { .content .post .post-meta .date .posted-on { margin-left: 0; margin-right: 1.5rem; } + .content .post .post-meta .tags .tag { + display: inline-block; + padding: 0.3rem 0.6rem; + background-color: #e0e0e0; + border-radius: 0.6rem; + line-height: 1.4em; } + .content .post .post-meta .tags .tag a { + color: #212121; } + .content .post .post-meta .tags .tag a:active { + color: #212121; } .content figure { margin: 0; padding: 0; } @@ -248,9 +3331,9 @@ img.emoji { .list ul li .date { display: inline-block; flex: 1; - width: 20.0rem; + width: 20rem; text-align: right; - margin-right: 3.0rem; } + margin-right: 3rem; } @media only screen and (max-width: 768px) { .list ul li .date { display: block; @@ -259,7 +3342,7 @@ img.emoji { font-size: 1.8rem; flex: 2; color: #212121; - font-family: Lato, Helvetica, sans-serif; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", Helvetica, "游ゴシック", "PingFang SC", STXihei, "华文细黑", "Microsoft YaHei", "微软雅黑", SimSun, "宋体", Heiti, "黑体", sans-serif; font-weight: 700; } .list ul li .title:hover, .list ul li .title:focus { color: #1565c0; } @@ -275,18 +3358,18 @@ img.emoji { .centered .about { text-align: center; } .centered .about h1 { - margin-top: 2.0rem; + margin-top: 2rem; margin-bottom: 0.5rem; } .centered .about h2 { - margin-top: 1.0rem; + margin-top: 1rem; margin-bottom: 0.5rem; font-size: 2.4rem; } @media only screen and (max-width: 768px) { .centered .about h2 { - font-size: 2.0rem; } } + font-size: 2rem; } } .centered .about ul { list-style: none; - margin: 3.0rem 0 1.0rem 0; + margin: 3rem 0 1rem 0; padding: 0; } .centered .about ul li { display: inline-block; @@ -294,41 +3377,109 @@ img.emoji { .centered .about ul li a { color: #212121; text-transform: uppercase; - margin-left: 1.0rem; - margin-right: 1.0rem; + margin-left: 1rem; + margin-right: 1rem; font-size: 1.6rem; } .centered .about ul li a:hover, .centered .about ul li a:focus { color: #1565c0; } @media only screen and (max-width: 768px) { .centered .about ul li a { font-size: 1.4rem; } } - .centered .about ul li a i { - font-size: 3.2rem; } .centered .error { text-align: center; } .centered .error h1 { - margin-top: 2.0rem; + margin-top: 2rem; margin-bottom: 0.5rem; font-size: 4.6rem; } @media only screen and (max-width: 768px) { .centered .error h1 { font-size: 3.2rem; } } .centered .error h2 { - margin-top: 2.0rem; + margin-top: 2rem; margin-bottom: 3.2rem; font-size: 3.2rem; } @media only screen and (max-width: 768px) { .centered .error h2 { font-size: 2.8rem; } } +.notice { + border-radius: 0.2rem; + position: relative; + margin: 2rem 0; + padding: 0 0.75rem; + overflow: auto; } + .notice .notice-title { + position: relative; + font-weight: 700; + margin: 0 -0.75rem; + padding: 0.2rem 3.5rem; + border-bottom: 1px solid #fafafa; } + .notice .notice-title i { + position: absolute; + top: 50%; + left: 1.8rem; + transform: translate(-50%, -50%); } + .notice .notice-content { + display: block; + margin: 2rem 2rem; } + +.notice.note { + background-color: #7e57c21a; } + .notice.note .notice-title { + background-color: #673ab71a; } + .notice.note .notice-title i { + color: #5e35b1; } + +.notice.tip { + background-color: #26a69a1a; } + .notice.tip .notice-title { + background-color: #0096881a; } + .notice.tip .notice-title i { + color: #00897b; } + +.notice.example { + background-color: #8d6e631a; } + .notice.example .notice-title { + background-color: #7955481a; } + .notice.example .notice-title i { + color: #6d4c41; } + +.notice.question { + background-color: #9ccc651a; } + .notice.question .notice-title { + background-color: #8bc34a1a; } + .notice.question .notice-title i { + color: #7cb342; } + +.notice.info { + background-color: #42a5f51a; } + .notice.info .notice-title { + background-color: #2196f31a; } + .notice.info .notice-title i { + color: #1e88e5; } + +.notice.warning { + background-color: #ffca281a; } + .notice.warning .notice-title { + background-color: #ffc1071a; } + .notice.warning .notice-title i { + color: #ffb300; } + +.notice.error { + background-color: #ef53501a; } + .notice.error .notice-title { + background-color: #f443361a; } + .notice.error .notice-title i { + color: #e53935; } + .navigation { - height: 6.0rem; + height: 6rem; width: 100%; } - .navigation a, .navigation span { + .navigation a, + .navigation span { display: inline; - font-size: 1.6rem; - font-family: Lato, Helvetica, sans-serif; - font-weight: 700; - line-height: 6.0rem; + font-size: 1.7rem; + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", Helvetica, "游ゴシック", "PingFang SC", STXihei, "华文细黑", "Microsoft YaHei", "微软雅黑", SimSun, "宋体", Heiti, "黑体", sans-serif; + font-weight: 600; color: #212121; } .navigation a:hover, .navigation a:focus { color: #1565c0; } @@ -342,8 +3493,8 @@ img.emoji { margin-top: 0; } @media only screen and (max-width: 768px) { .navigation .navigation-list { - position: absolute; - top: 6.0rem; + position: relative; + top: 2rem; right: 0; z-index: 5; visibility: hidden; @@ -351,9 +3502,9 @@ img.emoji { padding: 0; max-height: 0; width: 100%; - background-color: #FAFAFA; - border-top: solid 2px #E0E0E0; - border-bottom: solid 2px #E0E0E0; + background-color: #fafafa; + border-top: solid 2px #e0e0e0; + border-bottom: solid 2px #e0e0e0; transition: opacity 0.25s, max-height 0.15s linear; } } .navigation .navigation-list .navigation-item { float: left; @@ -363,18 +3514,20 @@ img.emoji { .navigation .navigation-list .navigation-item { float: none !important; text-align: center; } - .navigation .navigation-list .navigation-item a, .navigation .navigation-list .navigation-item span { - line-height: 5.0rem; } } - .navigation .navigation-list .navigation-item a, .navigation .navigation-list .navigation-item span { - margin-left: 1.0rem; - margin-right: 1.0rem; } + .navigation .navigation-list .navigation-item a, + .navigation .navigation-list .navigation-item span { + line-height: 5rem; } } + .navigation .navigation-list .navigation-item a, + .navigation .navigation-list .navigation-item span { + margin-left: 1rem; + margin-right: 1rem; } @media only screen and (max-width: 768px) { .navigation .navigation-list .separator { display: none; } } @media only screen and (max-width: 768px) { .navigation .navigation-list .menu-separator { border-top: 2px solid #212121; - margin: 0 8.0rem; } + margin: 0 8rem; } .navigation .navigation-list .menu-separator span { display: none; } } .navigation #dark-mode-toggle { @@ -389,7 +3542,7 @@ img.emoji { display: none; } @media only screen and (max-width: 768px) { .navigation #menu-toggle:checked + label > i { - color: #E0E0E0; } + color: #e0e0e0; } .navigation #menu-toggle:checked + label + ul { visibility: visible; opacity: 1; @@ -398,8 +3551,8 @@ img.emoji { display: none; } @media only screen and (max-width: 768px) { .navigation .menu-button { + position: relative; display: block; - margin: 1.8rem 0; font-size: 2.4rem; font-weight: 400; } } .navigation .menu-button i:hover, .navigation .menu-button i:focus { @@ -411,9 +3564,9 @@ img.emoji { color: #1565c0; } .pagination { - margin-top: 6.0rem; + margin-top: 6rem; text-align: center; - font-family: Lato, Helvetica, sans-serif; } + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen-Sans, Ubuntu, Cantarell, "Helvetica Neue", Helvetica, "游ゴシック", "PingFang SC", STXihei, "华文细黑", "Microsoft YaHei", "微软雅黑", SimSun, "宋体", Heiti, "黑体", sans-serif; } .pagination li { display: inline; text-align: center; @@ -429,11 +3582,75 @@ img.emoji { text-align: center; width: 3.2rem; } +.tabs { + display: flex; + flex-wrap: wrap; + margin: 2rem 0 2rem 0; + position: relative; } + .tabs.tabs-left { + justify-content: flex-start; } + .tabs.tabs-left label.tab-label { + margin-right: 0.5rem; } + .tabs.tabs-left .tab-content { + border-radius: 0px 4px 4px 4px; } + .tabs.tabs-right { + justify-content: flex-end; } + .tabs.tabs-right label.tab-label { + margin-left: 0.5rem; } + .tabs.tabs-right .tab-content { + border-radius: 4px 0px 4px 4px; } + .tabs input.tab-input { + display: none; } + .tabs label.tab-label { + background-color: #e0e0e0; + border-color: #ccc; + border-radius: 4px 4px 0px 0px; + border-style: solid; + border-bottom-style: hidden; + border-width: 1px; + cursor: pointer; + display: inline-block; + order: 1; + padding: 0.3rem 0.6rem; + position: relative; + top: 1px; + user-select: none; } + .tabs input.tab-input:checked + label.tab-label { + background-color: #fafafa; } + .tabs .tab-content { + background-color: #fafafa; + border-color: #ccc; + border-style: solid; + border-width: 1px; + display: none; + order: 2; + padding: 1rem; + width: 100%; } + .tabs.tabs-code .tab-content { + padding: 0.5rem; } + .tabs.tabs-code .tab-content pre { + margin: 0; } + +.taxonomy li { + display: inline-block; + margin: 0.9rem; } + +.taxonomy .taxonomy-element { + display: block; + padding: 0.3rem 0.9rem; + background-color: #e0e0e0; + border-radius: 0.6rem; } + .taxonomy .taxonomy-element a { + color: #212121; } + .taxonomy .taxonomy-element a:active { + color: #212121; } + .footer { width: 100%; text-align: center; - line-height: 2.0rem; - margin-bottom: 1.0rem; } + font-size: 1.6rem; + line-height: 2rem; + margin-bottom: 1rem; } .footer a { color: #1565c0; } @@ -444,22 +3661,27 @@ img.emoji { position: fixed; font-size: 1.6em; } .float-container a { - display: block; - text-align: center; - width: 3.0rem; - height: 3.0rem; + position: relative; + display: inline-block; + width: 3rem; + height: 3rem; + font-size: 2rem; color: #000; - background-color: #E0E0E0; - font-size: 2.0rem; - border-radius: 0.5rem; - opacity: 50%; - transition: all .25s ease-in; } + background-color: #e0e0e0; + border-radius: 0.2rem; + opacity: 0.5; + transition: all 0.25s ease-in; } .float-container a:hover, .float-container a:focus { color: #1565c0; - opacity: 100%; } + opacity: 1; } @media only screen and (max-width: 768px) { .float-container a:hover, .float-container a:focus { color: #000; - opacity: 50%; } } + opacity: 0.5; } } + .float-container a i { + position: absolute; + top: 50%; + left: 50%; + transform: translate(-50%, -50%); } /*# sourceMappingURL=coder.css.map */ \ No newline at end of file diff --git a/static/images/colt5_transformer_layer.png b/static/images/colt5_transformer_layer.png new file mode 100755 index 0000000..6632216 Binary files /dev/null and b/static/images/colt5_transformer_layer.png differ diff --git a/static/images/klu-gqa-grouped-query-attention.webp b/static/images/klu-gqa-grouped-query-attention.webp new file mode 100755 index 0000000..6ad0820 Binary files /dev/null and b/static/images/klu-gqa-grouped-query-attention.webp differ diff --git a/static/images/t_global_attention.png b/static/images/t_global_attention.png new file mode 100755 index 0000000..7acc012 Binary files /dev/null and b/static/images/t_global_attention.png differ