Hello, Guest the thread was called349 times and contains 18 replays

last post from mega65 at the

Networking support on Mega65

  • Heyas!


    I'm currently developing a networked (client/server) version of Yahtzee! and a client for the C64. This is just a test for me and I'm hoping to develop other networked games, too.


    Presently, I can only support the RR-Net and ETH64 devices.


    There was some talk once, long ago, about emulating the RR-Net (?) in C64 mode on the Mega65. I'm wondering if this is still the case? I was initially against the idea but I've changed my position and think this would probably be a good idea.


    I'm really hoping that my games can run on the Mega65. For technical details, I'm using the ip65 library with ca65 to build the programs.



    Daniel.

  • Hi Daniel, we are at the moment in the process to develop the network capabilities. I can't give further details, since i am not involved, but i will forward your question and ask the team to give you an answer.

  • Howdy,


    Super to hear that you would like to add MEGA65 support to your networked games. We totally get that this is a "first step", and agree that starting with something relatively simple is a good way to go.


    We dropped RR-NET compatibility on the MEGA65, because it turned out to be a horror to implement all the weird buffer handling that the chip it uses requires when in 8-bit mode. Instead, we opted for a very simple and much more performant design.


    Basically we have a 2KB TX and 2KB RX buffer, and a much simpler control interface. We are still developing the documentation, but an initial dump of stuff:


    First, these are the (current and possibly subject to change) registers. These all require you be in MEGA65 IO mode (so write $47 to $D02F, and then $53 to $D02F to open this IO mode).


    GS $D6E0.0 Clear to reset ethernet PHY

    GS $D6E0.1-2 Read ethernet TX bits currently on the wire

    GS $D6E0.3 Read ethernet RX data valid

    GS $D6E0.4 Allow remote keyboard input via magic ethernet frames

    GS $D6E0.6 Indicate if ethernet RX is blocked until RX buffers rotated

    GS $D6E0 Ethernet control

    GS $D6E1.0 reset ethernet PHY

    GS $D6E1.1 - Set which RX buffer is memory mapped

    GS $D6E1.2 - Indicate which RX buffer was most recently used

    GS $D6E1.2 WRITE ONLY Enable real-time CPU/BUS monitoring via ethernet

    GS $D6E1.3 Enable real-time video streaming via ethernet (or fast IO bus if CPU/bus monitoring enabled)

    GS $D6E1.4 - Ethernet TX IRQ status

    GS $D6E1.5 - Ethernet RX IRQ status

    GS $D6E1.6 - Enable ethernet TX IRQ

    GS $D6E1.7 - Enable ethernet RX IRQ

    GS $D6E1 - Ethernet interrupt and control register

    GS $D6E2 Set low-order size of frame to TX

    GS $D6E2 - TX Packet size (low byte)

    GS $D6E3 Set high-order size of frame to TX

    GS $D6E3 - TX Packet size (high byte)

    GS $D6E4 = $00 = Clear ethernet TX trigger (debug)

    GS $D6E4 = $01 = Transmit packet

    GS $D6E4 Ethernet command

    GS $D6E5.0 - Ethernet disable promiscuous mode

    GS $D6E5.1 Disable CRC check for received packets

    GS $D6E5.2-3 Ethernet TX clock phase adjust

    GS $D6E5.4 Accept broadcast frames

    GS $D6E5.5 Accept multicast frames

    GS $D6E6.0-4 - Ethernet MIIM register number

    GS $D6E6.7-5 - Ethernet MIIM PHY number (use 0 for Nexys4, 1 for MEGA65 r1 PCBs)

    GS $D6E7 - Ethernet MIIM register value (LSB)

    GS $D6E8 - Ethernet MIIM register value (MSB)

    GS $D6E9-E - Ethernet MAC address

    GS $D6EF - DEBUG show current ethernet RX state


    The packet buffer itself is visible at $FFDE000-$FFDEFFF. There are 2 x 2KB RX buffers, so that you can be processing a buffer while another packet is being received. The TX buffer is a little weird, because it is mapped write-only at the same address as the RX buffer (which is read-only). So if you read, you get from the RX buffer, and if you write, you put in the TX buffer.


    Sending a packet is super easy: Write the packet into the TX buffer, the length into $D6E2/3, and then $01 into $D6E4.


    Receiving, you just have to make sure that the buffers are flipped the right way around to allow receiving of a packet: If bit 6 of $D6E0 is set, you need to flip the RX buffers by XORing $D6E1 with $02.


    IRQs should work fine for both RX (packet received) and TX (packet sent, and next packet can now be sent).

    Overall, the whole setup was made to be eas easy as possible to use.


    You can read the MIIM information if you want to know if there is an ethernet cable connected etc. You should probe all MIIM PHY numbers until you get a response (ie. values not being $FF when reading an MIIM register value).


    Setting up a MAC address in $D6E9-E would be sensible, but only if it hasn't already been pre-populated for you (the MEGA65 Hypervisor should put a sensible value in here).


    In comparison with the RR-NET interface, the MEGA65's ethernet is MUCH MUCH faster. First, it is 100mbit, not 10mbit. Second, because you can DMA packets in and out of the buffers, and have working IRQs, it is quite possible to send/receive several MiB per second.


    Apart from that, there really isn't very much to it, other than that we are currently fixing some bugs with it all.


    It would be great to implement an IP65 driver for this, and to then commit it up-stream to make it easier for everyone to support in future.


    Do you have real hardware on which to test, or are you using Xemu's MEGA65 emulator, or what are your plans here?


    Paul.

  • Xemu/mega65 has *some* not so nice support to emulate of ethernet controller of the mega65, but it has its problems. First, there is currently some nasty locking issues (it needs to run another thread), and no IRQ is supported (since it seems nobody use this yet, just for polling). Second, since it's low level ethernet it works only with linux, and even there there is some network knowledge needed to set-up a "tap" device with correct rights, routing prepared, iptables, ip forwarding, DHCP setup etc, so generic basic networking must be known (I assume those as generic, maybe not so much for others ...). I'm not aware that Windows has similar tap devices as linux has, btw, but I don't know windows too much, that's for sure. Btw once I've started to document this here: https://github.com/lgblgblgb/xemu/wiki/Xemu-Mega65-ethertap


    Oh, one thing I forgot: recently mega65 has some changes in RX buffer handling, I'm not sure yet about its status in mega65-core, but certainly Xemu implemented the situation BEFORE this change. Once things are sorted out (for me ...) how it works on a real mega65, I can try to implement that in xemu/mega65 as well, of course.

  • Yes, this information will be very useful!


    We will need to implement a driver in the ip65 library. Unfortunately, I'm not the greatest network layer programmer so I will have to put the Mega65 compatibility on hold for the moment. This could be something I look at in the near future, after I finish the client but my Nexys4 board is out of action right now. I have to set it all up again with the new software and core. It all takes time and its such a limited resource. I'm assuming that the networking will function on the Nexys4 as well as the actual machines?


    As for the networking in Xemu only working on Linux, perhaps you could look at how VICE does it for Windows and use that? I forget how it works exactly but you get some kind of bridge drivers and then connect to those from the software. I think its a little nasty but it does work.


    Ahh yes after a quick look, Winpcap is what you use but its been superseded by something else now. I forget. It wasn't hard to find when I got it working, though.



    Daniel.

  • Yes, this information will be very useful!


    We will need to implement a driver in the ip65 library. Unfortunately, I'm not the greatest network layer programmer so I will have to put the Mega65 compatibility on hold for the moment. This could be something I look at in the near future, after I finish the client but my Nexys4 board is out of action right now. I have to set it all up again with the new software and core. It all takes time and its such a limited resource. I'm assuming that the networking will function on the Nexys4 as well as the actual machines?

    Actually I've already did that, I mean the IP65 + Mega65, I could even use the telnet app (as an example from IP65) on Mega65 to telnet to my Linux PC from Mega65. Though it seems to have some major problems, and also, the mentioned status of ethernet in mega65-core makes it hard to see, what the "stable" solution can be. So I'm working on that too, just for now, as with Xemu/Mega65 + Ethernet support as well, I would wait to have for mega65-core with the ongoing ethernet changes, which makes incompatibility issues currently with the older solution. I even have my half-finished iRC client code utilizing IP65, using some Mega65 features over std C64 features, like 80 column screen, DMA driven screen scrolling for faster reaction and so on. (and for sure the much higher clock speed so now Mega65 responds to ICMP echo request - commonly called only "ping" - faster than my router on my home LAN).


    I also have the nexys4 board, and testing on it, btw. So, when I say "I tested it on Mega65" I mean about the Nexys4 board. As far as I know there should be no difference in relation of Ethernet ...

    Quote

    As for the networking in Xemu only working on Linux, perhaps you could look at how VICE does it for Windows and use that? I forget how it works exactly but you get some kind of bridge drivers and then connect to those from the software. I think its a little nasty but it does work.

    You're right, that was my idea too, to have a look how others do that on Windows :) The main problem, that any windows thing is slower for me, since I'm not even a windows user since windows 3.1 (!!!!) I have no windows at all. Thus I have to learn many windows related programming just because of Xemu, otherwise I don't need windows (either just user or programming) knowledge/experience at all (not in my personal life, not in my official work either, being UNIX engineer ...). But I see the problem if I want to create something useful and most people want to use Windows, it's something I can't simply deny but I have to try to adopt to the needs ... But it takes time, also this is not the work I'm living from, so it can a bit slow process :(

  • Btw, if UDP is enough, it's much more mature and compact solution to use your own IP/UDP implementation. Well, if TCP is involved as well, it's a harder thing, and I rather would not write my own implementation but rely on IP65 for example, or maybe uIP, or whatever. But for strict UDP stuff, and no TCP it's kinda easy to do and only few hundreds of bytes, while IP65 is quite sized and maybe not so even nice for your needs (eg how it works the polling model, exact callbacks etc). Once the whole ethernet issue on M65 settles down, and anybody is interested in a light-weight UDP only "library" or so, I may open mine to the public. But again, SURELY for a "full" tcp/ip suite, some wants to use things like IP65, for sure!!!!! For IP65, I already has my mega65 ethernet driver (I guess the same for Contiki, since the same driver "format"), but again, currently the same problem: there are multiple versions of mega65-core with different maturity of the ethenet, so some should wait till it's settled down to a more or less stable state :)


    But, if you're serious, and you would like some help, you can write me in private too, of course, I'm always happy if I can help (and maybe I can learn something too meanwhile :) ).

  • Awesome that you have an ip65 driver in the works already. I will definitely want to get that from you in the not too distant future and will very likely need some help with it.


    As for UDP... I think its fine if you don't need to know for sure that the client "is there" all the time and if you're doing continuous state updates that needn't be acknowledged all of the time but for my Yahtzee! client, I really do need to know for sure that the client is still connected. Also, there is only a state update when one of the users does something and it has to be acknowledged and further, these updates have to occur in the correct order (mostly). TCP is the best for it.


    Yes... There is a huge overhead in processing, code and data. I think that there is about 4KB (at least) of code and another 4KB of data required by ip65 to get TCP/IP working. I can't remember how much effective raster time it uses/needs but I think its a lot. It hurts a lot for my client because I have a fairly complex interface controls system (to make it nice for the user) which also "chews" much memory. It does allow me to update things more easily while I'm polling the TCP/IP driver though. I'm up to about 24KB in total usage and I have only a skeleton, most of the interface and the start of some processing functions but not much specific message handling although I can make the connection to the server and do its "handshaking". I will be able to complete the client, I'm certain but I am a bit dismayed. I am concerned about getting in a sound and music driver both for how much memory it needs and for the processing time though. It would be a shame to go without but we'll have to see.


    Since you mentioned it, there is a possibility I should have made it a Contiki based program but I haven't been bothered trying to get it to work with ca65 in the build chain. I really don't like C and C++ and I honestly don't want to be bothered with learning how to get cc65 working with code I want to write in assembly. I guess I could be a little lazy but I did want the entirety of the code to be in assembly. I don't mind effectively writing my own controls library. I have much bigger plans in mind and it needs to be as efficient as possible... This "practice run" has been really insightful.


    If you want to check out the client, I have it uploaded on GitHub at the M3wP Yahtzee! GitHub repository.


    I usually have a server running and can give you the address in private if you would like to try but the game isn't playable on the C64 yet. If some nice person has Delphi 10.1 (or is it 10.2? I forget) they they can compile a Linux version of the client and server for us... I haven't been able to justify the expense of upgrading (I am considering it for the next 6 monthly budget ^^) but I needed to use FMX to make a nice Android compatible client. Still, whats there is there and you can see my source code, too.


    Whew! This post has gotten long. I won't do a TLDR section, I hate them :P



    Daniel.

  • Haha, TLDR is not my thing either, so be my guest ;-P Surely, I don't want to keep those code to myself at all! Just as I've told, it's pointless to distribute now, when it does only work with a specific bitstream for mega65, maybe cause more headache for people than not having anything :-O As soon as things seems to be working surely it will be available, in fact, I have the suspect, the best way for me to contact ip65 repository authors to include my driver so it's "built-in" then at the official repository, no need to dig for mega65 support elsewhere ;-P


    For UDP, even many "commercial and modern" stuff uses UDP, inventing its own acknowledgement mechanism built-in the protocol they use, but over UDP. My main concern about TCP, that it's far too complicated, like fragmentation, window sizes, especially mixed in with modern ages networking, then window scaling, ECN, side-channel ICMP messages etc etc can cause problems. Surely, most modern OSes has no problem to have support all of these (though even modern firewalls sometimes have problems do not handle those well, or don't accept ICMP as a relation-state of a TCP connection) but for a resource limited scenario it would be horrific to support all of the TCP things well enough. My experience with IP65+MEGA65 and using with connection to my Linux box that sometimes it slows down a lot, an according to tcpdump many odd things happens, what Linux thinks about the connection. Surely Linux would treat the client being a modern TCP implementation more or less, and trying insane things to enhance the tcp communication which generally speaking is good, but mega65 won't be a partner of that, having only a minimal TCP implementation on it, in the name of IP65 ... One of the things I still don't understand too much, that it works mostly, but if there is a burst of heavy communication it starts to log more and more, sometimes needs minutes (!) to recover. I still cannot find the problem even with tcpdump, it seems mega65 tries to re-transmit ACKs several times but not so much happening at my Linux side, it simply waits even minutes to react. I guess it may treat the party "too slow" or other problem and by intent it delays everything not to waste energy for a slow client :-O Sorry for the not-so-professional kind of description of the problem, this is my best I can do in English, I'm afraid :) But probably this can be even an ethernet issue still on the mega65 side waiting to be fixed (maybe not every frames are TX'ed I want, and there can be RX problem as well to miss some ethernet frames), and there was a CIA implementation bug discovered by me as well in relation of 16 bit timers what IP65 uses as well to "measure time" for waiting/polling/re-transmissions/etc ... That's one reason, I would say, it's not so wise RIGHT NOW to start to build things on my driver, since things may change. And in fact already did with RX buffer flip already.

  • Yes, I agree there are many issues and this is why I'm starting with something "simple" although "real world" and seeing how it goes.


    I don't want to implement my own ACK/NAK, keep alives and so on. That's what TCP is for! Hehe :) The packets really have to be in the correct order too and this isn't guaranteed with UDP. Its really a test to see what's possible. I do agree that you should probably try to use UDP where possible, though.


    I have noticed fragmentation and message "condensation" (multiple messages in one packet) in my "PC" (Windows/Android) client and server and have done my best to handle that. I have markers (a length byte) in the messages to help determine if a whole message or more is being received.


    I don't know what to do about "side channel" and ICMP. I'm hoping this is handled in the driver and its just ignored. I think the C64 client could easily be swamped by a malicious server though.


    I think I should get away with it all, in the end. I'm not going to try to implement the server on the C64. I think that is really out of scope.



    Daniel.

  • Surely, if you need a proper "safe"/etc connection, TCP is designed that for, no doubts here :) Just especially things like in networked games (at least what I can think now, there can be tons of different scenarios), it's more like a rapid exchange of (smaller amount of) information from time to time without the hassle of a whole TCP connection when you need to even keep-alive etc, with lots of code, when UDP is much more better for this kind of thing. Surely it depends on the needs. For example this UDP theory can be even applied for streaming better even on Mega65 and on the "modern Internet" as well (and then it's quite possible to peak M65's possibilities to stuff in the 100mbit ethernet hehe).