basic+ documentation of common functionality

[GlenDC]

It would be great if the common functionality and stuff used in the more common use cases are either self-documented or refer to documentation that help explain it. I think the README and CLI help documentation that's there is already a great start.

A big chunk of this project are the collection of other projects, so for anything that that uses such code directly a reference to its documentation (for that specific part) would be sufficient, sometimes a hybrid thing will be required. E.g. it would be nice to have a list of all objects exposed in the REPL, I think they already are, and for those it would be sufficient (I think) to reference the Puppeteer documentation for those object locations. Where extended it would require some extra documentation.

Important for me, and its motivation, are locations such as the interceptor where it is not very clear to me what possible values are for the ResourceType and RequestStage. For now it seems to be only possible to have Response as the requestStage, and I haven't seen any example not using Script for resource type. However for a purpose where I want a resource other than a Script I omit it for now, given I do not know what other resource type I can use. Omitting it seems to work as well (I guess it allows any resource type?). I tried GREPping through the code base and it doesn't seem to be something this codebase really defines, so I imagine it is another piece of software glued into here. Which is ok as well of course, if so, a reference to the documentation about those parameters would suffice as well.

And there are probably other examples that fall in the category of the examples I gave here.

I wouldn't mind contributing this myself, I would just need some pointers in the different pieces of software used and where/how they are used.

[jsoverson]

Definitely, thanks for pointing this out. I've started fleshing out documentation locally and will finish it up ASAP.

Feel free to ask questions in this issue and I'll address them as I go.

[GlenDC]

Thank you for that! Very much appreciated.

Besides what I already mentioned, I was also a bit stuck in getting started with the entire shift-tool suite (e.g. refactor and query) given those help a lot with the kind of interceptors I might want to write.

Your excellent tutorial series at https://github.com/jsoverson/deobfuscating-js-streams already clarifies a lot with that.
First I was following with you, but since stage3 I try to do it on my own first and than compare, which seems to work out good so far. With it I also have a lot of extra comments and notes with stuff I learned about each part of the script while writing and afterwards listening. And the code is up to date with the latest version of your tools. I wonder if somehow you want me to contribute that back to that tutorial repository, so the code is up to date and has extra helpful comments. Or perhaps you would rather not.

---

A question I do have for now is the following.

It's about something that so far is not clear to me. Perhaps you cover that in stage 5, I am only at stage 4 so far. Is that I wonder if it is already execute some part of the code so that I can stay as flexible as possible. I know you touched on it a bit early in the series but that's from a year ago and still a pipeline dream at that point. I wonder if that's already a possibility. Talking about code such as:

(function(a, b, c){
   // ... do stuff that injects functions as methods of some of x, y and z
})(x, y, z);

Such that I can than continue further with x, y, z, already having some of the logic done.

[jsoverson]

Updated the readme with more information. It doesn't add enough, but it's a start. The ways to use this project are too big for a single readme file and I need to separate them.

I'm not 100% clear on what you are referring to in your question. Those streams are from a long time ago and it may be possible with shift-refactor and shift-interpreter but I'm not 100% clear on how to answer. I just created an AMA repository for questions like this, feel free to repost there: https://github.com/jsoverson/javascript-deobfuscation-AMA

[GlenDC]

I'm not 100% clear on what you are referring to in your question.

It's Okay, I'm going to play with it a bit more when I get around to it. If I do not find an approach that satisfies me I'll definitely reach out. Glad to know this repo exists, I will definitely, very much so, keep it in mind :)

---

The README does look a lot better already indeed. I can understand that indeed fitting it all in the README will be hard once you expand sufficiently in your explanations. I guess your explanations related to the interceptor configuration and such are still WIP. No problem though, I'm already happy this project exists, so once agains a big thanks though!

---

I know the streams are from a long time ago, but they definitely did help a lot in understanding how I could use tools such as shift-reactor within a hackium interceptor, without those streams I was pretty lost in that part. If nothing else they explain a lot about the philosophy behind it all and how to approach these interception problems.