hello everyone, for some background I’m a rookie dev who’s new to homebrew and OSS development and was interested in contributing to pretendo, so recently I’ve done as much research and preparation as I could to figure out how I could get started doing so, which has left me with some questions that I couldn’t really figure out the answers to. I apologize in advance if anything I ask could’ve easily been answered had I done just a bit more digging, I’m just having a hard time navigating through everything at the moment
I was originally intending on contributing to pretendo’s implementation of NEX since I saw on the progress meter on the website (which I’m now led to believe is at least a little bit outdated) that NEX was the least developed part of the stack, and after I familiarized myself with the structure of the codebase and skimmed through some of it, it seemed to me that most of the framework was either already implemented or was at least set up for future development, and the issues I saw on github conveyed to me that the main thing that’s being worked on are mostly bug fixes. what I was wondering is if there were any parts of NEX or services that used it are still in early development or haven’t been implemented yet as at the moment I’m not sure how well I could contribute with bug fixes and the like, I do feel like I would be more comfortable following documentation to implement any features that haven’t been yet (and on that note, I’m open to help contribute to other things besides NEX that might be early in development)
on a related note, I might be getting ahead of myself with trying to figure out what I can help with as I have yet to set up a workflow for developing anything. I used the pretendo-docker github repo to setup and connect to a local account server but was not able to figure out how to set up and direct my wii u to a local NEX server, and even then I’m generally confused as to how NEX (or possibly other parts of the stack) would get developed and tested on real hardware after that’s set up and would like some insight on what exactly happens there
any comments are appreciated, thank you very much for reading
I apologize if you already know this but it wasn’t totally clear from your post. NEX is a library that the game servers use, and by using pretendo-docker you are already connecting to local NEX servers with Friend List, Super Mario Maker, etc. I would say that contributing to a low-level networking library like that is probably not the easiest way to get started with contributing, but you can check Documentation | Nintendo Wiki to learn more about NEX.
@Wolftastic is a recent new contributor to NEX, he might have some thoughts to add on how to get started.
On my end, pretendo-docker is unfortunately not a great development environment for contributing yet. While the servers run, I haven’t documented how to contribute with it, and the fact that it uses manual patches for many of the servers would cause extraneous changes when one commits. Fixing this is on the roadmap.
hey, thanks a lot for the reply! I was aware of some of the things you brought up, but I’m glad that you (as the creator of the repo) were able to clear up the misunderstandings I had about pretendo-docker. I had chosen to use it precisely because there was no official self-hosting setup, I have never (for anything else other than minecraft lol) set up a server before, and I knew enough about docker. in that case I would need to figure out an alternative server setup that would make it easier to develop for NEX if I still end up going down that path.
Yeah, I’m a fairly new contributor to the NEX stuff. I think it’s the best place to make a big impact, and is the place which has the most documented specific issues at the moment.
The work I’ve done so far has been by using the nex-go library to write dummy servers and making requests against them with nintenodclients in an attempt to reproduce bugs described in the nex-go issues on GitHub. That’s been pretty successful so far and has helped me learn a whole lot about how nex works (I’d also never used Go before, so it was a way for me to learn that)
It might sound a little boring but I think you’ve got the right attitude at the moment. Most of this stuff is quite complex and in-depth, so spending time reading documentation, looking at code, and experimenting with the libraries is all going to be very valuable to you
Another thing for you to look at is this issue which hasn’t been merged yet but is talking a lot about how to organise contributions and the different repositories too
If you’re on the discord there’s a lot of good chat on the dev/testing channels, although those are locked behind the supporter paywall
Not wanting to take this away from the forum, but if you have any questions about the nex stuff feel free to reach out on Discord, I’m more active there, but I do try to check the forums daily (not sure how I didn’t see this for 2 days, damn caching!)
thanks for reaching out! I originally hadn’t considered that workflow because I was under the impression that you primarily used nintendoclients if you didn’t have a console to work with as per the current contributing guidelines (not sure if that’s correct or not). ideally, I would’ve wanted to work with real hardware, but at the moment I’m more concerned with getting some work done on this if I can, so I’ll make sure to give it a shot. I’ll make sure to reach out if I’m having trouble with anything, thank you!
I unfortunately don’t have a real WiiU to test with (until the one I have on order arrives later in the week) also, a lot of the bugs that I’ve been working on would have been very difficult to reproduce on real hardware as they’re related to very specific circumstances (often race conditions). By just using nintendoclients, I can write extremely simple servers (literally just PRUDP in some cases, and extremely simple RMC in others) using nex-go and still run good tests
In this PR for example, the bug was related to a race condition that led to multiple different failure modes. This would have been almost impossible to reproduce reliably on hardware against a full blown game server. I ended up hammering my dummy server with 10000s of requests all at once to be able to consistently reproduce it
I’m not suggesting that this is the only way to work, or that this doesn’t have downsides. It absolutely does, for example when working on this issue it became clear that pings from nintendoclients are different to ping packets from most game clients as they are sent with the RELIABLE flag and share a sequenceID with other packets. So when we’re trying to make sure things work for real games on real hardware, there are definitely limitations here, but it’s a useful tool.
If I was working on something like a game server, I’d want to be testing it constantly with real hardware
PS I’d rather use something where I could include the tests in the repository themselves but nex-go has no client library, and the way it’s written makes that difficult to change (see this issue for more information)
Matthew and Wolf have already covered the gist of things, but just to chime in:
As Matthew said NEX is a library, which other libraries/servers consume. It’s the lowest level part of the game server stack. I agree that it might not be the best place to start, unless you know what you’re doing, but it is the best place to make big changes that affect large parts of the network.
NEX is a networking library used by Nintendo for their game servers. It provides a transportation layer, called PRUDP, and then many services (also called “protocols”) which expose many methods used for the actual online interactions. These service method calls use a protocol called RMC (Remote Method Call).
The consoles establish a UDP connection to a server using a custom transport layer called PRUDP, which adds reliability features ontop of UDP. These PRUDP connections send RMC payloads to and from the servers. These mechanisms together are called NEX
Nintendo did not make NEX from scratch, it’s based off another library called Rendez-Vous made by Canadian company Quazal. Nintendo licensed RDV from Quazal, modified it a bit, stripped some more complex features (like unreliable packets, compression, etc.), and rebranded it as NEX. A lot of the work we do is making sure nex-go is compatible with both Nintendo games and non-Nintendo (even non-Wii U/3DS) games, through reverse engineering the games which use these libraries. I use Xenoblade and WATCH_DOGS for NEX and RDV research respectively.
It’s a lot of work to contribute to, you need some good experience in networking and server management/development, and you’ll almost definitely need experience in reverse engineering (not super required in cases where issues are already layed out, but for finding new information you have to dig into the games).
thank you for the reply! at the moment I only have a really basic knowledge of networking (enough to follow the rundown and some of the documentation I’ve read) but was planning on learning more in the near future. you bring up that experience with servers and reverse engineering is super helpful for contributing, neither of which I have any real experience with. I know of some resources for learning about reverse engineering, but I wouldn’t know where to start with server dev/management, so I’ll be on the lookout
I should mention that one of the reasons I decided to start at NEX even though I don’t know a lot of things that I need to do so is that I couldn’t seem to find a pipeline to contribute to it as a more inexperienced dev if that makes sense, so it seemed simpler to just check it out, try what I can, and learn as I go, stopping to focus on learning if I got hard stuck on something. if there does exist one or something of the sort, I’d ideally want to go through that instead. in any case, I’m willing to learn whatever I need to potentially contribute in the future, even if it’s not much, since I think this is all really cool
That’s great to hear! I would say that the most approachable servers are the Node.js servers in Javascript or Typescript. Unfortunately, not everything that the devs want to work on gets publicly posted to the issue tracker, but you might be able to find a few approachable issues.
Also, very soon, there will be a new contributing guide. I suggest reading it as it is now, it’s still being reviewed before it’s merged as the official contributing guide but it’s pretty much finished at this point.
EDIT: it was just merged, see CONTRIBUTING.md.
One of the issues giving people a clear path to contribute is that there’s just such a broad variety of things to work on, and a lot of these topics are quite in depth/specialised. That said, I don’t think anything is beyond you if you’re interested, most things just won’t be immediately obvious. The other contributors are also friendly and are happy to answer questions, it’s just easiest to help when those questions are specific 
We’ve been working on making this kind of thing more accessible on GitHub. See here Plans · GitHub
thanks y’all! I’ll make sure to check out the servers and the new contributing guidelines as you mentioned Matthew. Wolftastic, your reply is definitely reassuring, and I’ve been able to see from this discussion that the community is indeed welcoming. I try to be sparing with my questions since I have a bit of trouble making good ones as I might’ve shown in this thread, but if I get really stuck I know to ask around
I’m all for being welcoming to newcomers, but I’ll also be realistic. Some of the areas we work in are just not really suited for inexperienced devs. There’s definitely some areas which are, but stuff like our lower level libraries probably aren’t. You’d probably have better luck checking out something besides the NEX libraries if you’re inexperienced?
of course, I’m open to working on other, higher-level parts of the stack, preferably ones where the skills I learn from working on them could potentially transfer over to working on the NEX stuff (I guess that’s my idea of a pipeline). I’ll have to do more research on the whole stack and see if there’s something better to do given my circumstances, and I’m definitely open to any suggestions. thank you!