Upcoming Textadept 12.0 alpha 2

[orbitalquark]

Heads up: I've committed a large set of changes to the Textadept and its individual modules repositories in anticipation of the next alpha in March. Please keep this in mind if you're pulling or downloading new nightly builds, and let me know if you encounter any issues. I'm sorry in advance :( This should be the last set of major changes before the next release. Notable changes include:

• Language modules are no longer auto-loaded. The auto-loading has been bugging me for some time, as they are an exception to the rule that all modules ought to be manually loaded. It is easy to add back auto-loading in your ~/.textadept/init.lua if you prefer having it. See https://orbitalquark.github.io/textadept/manual.html#language-module-changes
• The Language Server Protocol (LSP) module has been undergoing heavy development and should be better supported going forward. For now it remains an external module, but it may make it into the main repository sometime in the future.
• I have stopped maintaining all of the language modules I've been maintaining (C, Lua, CSS, HTML, Go, Python, Ruby). They have been removed from nightly builds. I was only using them for code completion and documentation, both of which are now in the domain of language servers, which now feel mature after the concept was introduced back in 2017. My goal is for the LSP module to replace the functionality in those individual modules.
• The LSP module now contains a simple Lua language server that is a replacement for some of the functionality in Textadept's old lua module. It is enabled by default for Lua files and provides code completion and documentation for Lua scripts as well as Textadept's API in the correct context. Feel free to turn it off by setting require('lsp').server_commands.lua = nil.
• Textadept can function as a Lua script interpreter using the -L or --lua flag.
• Lexers are once again invoked within Textadept's Lua state, but as part of a Lua library instead of as a Scintilla lexer. By using pure Lua, my goal is to eventually allow the use of Tree-sitter parsers for syntax highlighting and more.

For an overview of all of the changes from Textadept 11 so far, please see the running changelog and migration guide.

Once again, please let me know if you encounter any issues and I'll do the best I can to try and help. Thanks for your patience and understanding.

[rgieseke]

Textadept can function as a Lua script interpreter using the -L or --lua flag.

That's really cool! (I remember discussions about this a long time ago)

Does it mean all Textadept's built-in or loadable libraries are available?

[orbitalquark]

lfs and lpeg are available, but `package.path` does not contain Textadept's Lua modules, as they'd likely not be functional outside a Textadept environment. There's nothing preventing you from trying, though.

[rgieseke]

Great, thanks! (Will try!)

[eric1234]

My goal is for the LSP module to replace the functionality in those individual modules.

I haven't played with the LSP stuff yet but I really love it when I see a project reduce the scope of what it maintains while expanding what it can do and this decision looks like it might do that. Great work!

[snoopy]

allow the use of Tree-sitter parsers for syntax highlighting and more.

Very exciting. I am looking forward to that.

[paaguti]

That would be an overkill for TextAdept . It would cease to be an editor you can place in a VM because it is resource aware. Not to speak about all the external dependencies…Enviado desde mi iPhoneEl 14 feb 2023, a las 23:14, snoopy ***@***.***> escribió: allow the use of Tree-sitter parsers for syntax highlighting and more. Very exciting. I am looking forward to that. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[orbitalquark]

That would be an overkill for TextAdept . It would cease to be an editor you can place in a VM because it is resource aware. Not to speak about all the external dependencies…

What would be overkill? Using Tree-sitter? Or depending on language servers?

[paaguti]

Tree-sitter in general is bugging me a lot. I have tried enabling that in Emacs and the main headache is to get the language servers installed and running.

[orbitalquark]

Tree-sitter is a C library for parsing source code (like Scintillua). It doesn't need language servers at all. I have an experiment using Tree-sitter parsers with Scintilla that doesn't have any external dependencies: https://github.com/orbitalquark/tslexia

[paaguti]

Wow, the way you integrate the compilation process make it quite more attractive than what they do elsewhere... :-) Say I'd like to test it in TextAdept... two questions: 1.- Do I just compile everything into my .textadept/modules and then require('tslexia') ? 2.- What about Python? Would it be as easy as adding the tree-sitter Python parser? You have nearly 'converted' me :-)

…

On Wed, 15 Feb 2023 at 16:53, orbitalquark ***@***.***> wrote: Tree-sitter is a C library for parsing source code (like Scintillua). It doesn't need language servers at all. I have an experiment using Tree-sitter parsers with Scintilla that doesn't have any external dependencies: https://github.com/orbitalquark/tslexia — Reply to this email directly, view it on GitHub <#362 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZKU4EL47W6FG6WM5TJY7LWXT3YJANCNFSM6AAAAAAU2UJHUU> . You are receiving this because you commented.Message ID: ***@***.***>

-- Fragen sind nicht da um beantwortet zu werden, Fragen sind da um gestellt zu werden Georg Kreisler Headaches with a Juju log: unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run a leader-deposed hook here, but we can't yet

[orbitalquark]

Unfortunately there isn't a way to test it with Textadept. I just compiled TSLexia into a toy Scintilla application as a proof-of-concept. I hope to eventually come up with a solution like require('tslexia') in Lua. Since there is a Python Tree-sitter parser, it should "just work" when all of the pieces are in place :)

[paaguti]

Feedback on "the present" (just compiled TextAdept again):

• LSP is working nicely, will need to find a smooth keybinding, but that's my problem.
• [Minor] After editing a Python file, I get the following on the CLI:

~ ➤ textadept bin/xls2ical                                                     
QProcess: Destroyed while process ("pylsp") is still running.

• [Mild]: I at a black screen on my Linux until the LSP server is running. Not extremely annoying on Linux, yet to test on Windows and macOS.

For the rest, not having the stock modules for Python, C, etc. didn't collide with all my legacy init.lua stuff... :-)

[orbitalquark]

Thanks for the report.

I've also seen the "QProcess: Destroyed while process is still running" console warnings. Textadept does notify language servers to shut down, but the editor quits before they can I suppose. I'll see if I can find a way to wait for shutdown with a timeout.

During the LSP start process, Textadept needs to wait and see if startup and initialization was successful, hence the blank screen. I can't think of a way around this at the moment. It can be a bit annoying, I agree...

[orbitalquark]

After a bit of investigation, enabling logging for a language server shows that it received and processed the message, shutting down gracefully. Even if I try to sleep before quitting Textadept, I still get that QProcess message. I suspect this is an internal Qt thing with how it handles processes in multiple threads. Ultimately, I don't think this is an issue.

[ghost]

TextAdept Nightly. Edit -> Preferences to open init.lua will trigger [Output Buffer] with this error:

C:\textadept/core/lexer.lua:135: attempt to index a nil value (local 'buffer')
C:\textadept/core/lexer.lua:135: attempt to index a nil value (local 'buffer')

[orbitalquark]

This bug will be fixed in the next nightly build: 6701e81

[ghost]

Regarding LSP, I will very glad if TextAdept could support LSP over TCP. This language server only operates over TCP:

https://github.com/objeck/objeck-lsp

Currently there are only two options to use with it as I know: Sublime Text or VSCode 😮‍💨

[ghost]

orbitalquark/textadept-lsp#12

[ghost]

People tried their best to make their text editor's lexer more powerful to be able to correctly handle more complex languages. You are weird. You go away from your very own lexer and wanted to rely on the external language servers instead?

Let me give you an example: Netbeans IDE. I followed the tutorial to add support for a new language via the newly introduced LSP Client plugin. I have the textmate syntax file, so I expect it should be able to correctly handle the language even before I could make the language server works with it. It turned out to be wrong. The textmate syntax file is not enough. It only provides basic syntax highlighting. Code completion (basic ones like auto complete of brackets, double and single quotes,...) and auto indentation doesn't work. It relied on the language server to provide these features that I consider should be work out of the box without the need of any language servers. I ended up abandon the idea of using Netbeans IDE for that programming language. Do you want to make your text editor to be useless like that?

[orbitalquark]

Sorry for the confusion. Allow me to clarify Textadept's distinction between lexers and language servers.

Lexers are only for syntax highlighting. Textadept does not use lexers for any sort of smart language analysis, code completion, signature help, etc. If you write a lexer for Textadept, it will highlight your code. This is how it has been, and how it always will be.

Before LSP was invented in 2017, I created rudimentary code completion and signature help for C and Lua in 2011. It never utilized lexers. Over the years I expanded to CSS, HTML, Go, Python, and Ruby via language modules. Still, support was never very deep, relied on statically compiled lists of modules, classes, functions, etc., and was not very extensible.

Now fast forward to the present day where LSP and it's language servers are very mature. I am abandoning my rudimentary, outdated code completion and signature help routines in favor of the more sophisticated language servers that actually understand their respective languages at a deeper level than Textadept or its lexers ever could.

With regards to your Netbeans example, that sounds just like Textadept 11 and Textadept 12: if you give it a lexer, you will only get syntax highlighting and nothing else. (Textadept does give you language-agnostic auto-pairing of brackets and quotes via textadept.editing.auto_pair and basic auto-indent via textadept.editing.auto_indent, though.)