Hello, Guest the thread was called4.2k times and contains 40 replays

last post from ubik at the

MEGA65 ROM Benchmark Results

    A few days ago I received a ROM from adtbm for testing and review. In this ROM the MEGA team, first of all Bit Shifter , has optimized the BASIC 10 in the C65 ROM in parts, fixed some bugs and adapted it for a better use of the extended MEGA65 capabilities with BASIC 10.


    This resulting - I call it - "MEGA65 ROM" (because it doesn't work with the C65 emulator xc65 from LGB-Z anymore) is still in development and I'm glad that I was allowed to take a look at the current state 920116 already.


    I deliberately did not want to know in advance what has been changed, so that I do not program the small benchmark test unconsciously "optimized" for a particular result. ;)


    I promised adtbm that I would try to test as objectively as possible and independent of the team's wishes. At this point again a big thanks to adtbm for the new Porsche in my garage!


    To test the effects of the BASIC 10 optimizations made so far, I wrote a few short and unspectacular benchmark programs in BASIC 10 that test some typical "sticking points".


    The tests all ran on my Nexys board. On the one hand with the extended ROM 920116 and on the other hand as comparison with the latest C65-ROM 911210 from Commodore. :loet


    I also ran the benchmarks at 3.5 Mhz (POKE 0,64) and 40 MHz (POKE 0,65) respectively.


    I concentrated on the basics. So no graphics commands (except for one test that uses LOADIFF to read from disk image and display one IFF image per loop pass) or sound commands were included in the tests.


    These are the test programs. Nothing special, one or two for-next loops that run through one or two lines of test code a few times. At the beginning and at the end the timer TI$ is output and then I noted the difference in seconds in each case.





    The program "BENCH07" (loading a IFF image with LOADIFF) doesn't work with the new ROM 920116 at the Nexys board. It crashes trying to load the IFF:



    So I have no time result for this test for the new ROM.


    I summarize the results like this:


    With the new ROM 920116, all benchmarks are processed faster than with the 911210 ROM.


    The runtime was - depending on the test - from about 10% up to 47% faster.


    Just to put that in very clear terms: The faster processing is exclusively caused by the revised ROM. The exact same hardware was used for the test runs. Only the ROM file was changed.




    I can only say: Hats off to this great performance! :thumbup: If this continues with the optimizations, the 40 MHz mode will soon become obsolete. ;)



    At least for the moment the modification of the ROM for the MEGA65 will be a good solution for using the extensions of the MEGA65 (compared to the C65) also with BASIC 10.


    That in the medium to long term a generally better BASIC for the MEGA65 would be helpful and good (e.g. discussed here), remains a topic for itself.


    But at the moment and in the short term, this MEGA65 ROM will be the best BASIC 10 you can get for "no money". :)


    I will certainly keep an eye on the further development of the ROM and I am glad to be allowed to - perhaps - make a small contribution with the benchmark tests. If desired and I can accommodate it in time, I'm also happy to help again with advice and superfluous comments. :D


    Even if my personal focus - besides a BASIC 10 alternative - will still be the original C65 ROMs from Commodore (simply for historical reasons), I find the work on this modified ROM very gratifying and for the work with and on the MEGA65 a real progress. :thumbup:


    Even more so when it is implemented as well as has been done so far by Bit Shifter . My greatest respect for doing it! :thumbup::thumbup::thumbup:


    Keep it up, then you can also deliver a very great and more complete ROM with the MEGA65! :thumbsup:


    I have attached a zipped D81 disk image containing the benchmark programs and the IFF picture for BENCH07.

  • adtbm

    Added the Label Action required

    Hi Snoopy ,


    we are happy and proud that you like our ROM. Thank you !
    As you've noticed, the ROM is in constant development.


    Your benchmark is impressive ! i was blown away, when i've seen the numbers from your tests !

    Be asured, a more detailed answer is coming soon ;-)

    Hello Snoopy , I'm very pleased, that you wrote these benchmarks and published the results.
    I think, the inabilty to use LOADIFF is due to hardware differences between the Nexys board and the MEGA65.
    I test all changes on a MEGA65 developer computer and the emulator XEMU from LGB-Z .
    The new ROMs assume, that 256K RAM are available for BASIC, using bank 0 and 1 for program text/variables and bank 4 and 5 for graphics.
    This leads to ca. 56K free for program text and 56K for variables, while the 128K graphics RAM can store
    one hires screen with 640 x 400 resolution and 16 colours, or two screens with 320 x 200 and 256 colours.
    But I'll implement a test for the availability of these banks and print an error message, if for example LOADIFFF

    cannot be used due to missing RAM.
    Besides making BASIC faster, an important goal is to make BASIC10 stable and comfortable.

    Some examples:
    The ELSE BEGIN ... BEND bug, that was in BASIC since BASIC7 of the C128 is fixed.
    Typing RETURN on a line containing the READY. prompt doesn't produce an error message anymore.
    L shift O is again interpreted as LOAD and not LOADIFF, as in the C65 ROMs.
    And there is still more on the TODO list.
    Currently I clean up the code of the graphics system, making it faster, shorter and better documented.
    The other parts of BASIC will soon gain again speed by using the MEGA65 hardware multplication and division
    for index calculation and floating point math.
    It is fun working in this phantastic MEGA65 team, getting feedback, exchanging knowledge and working together.
    Best wishes,
    Bit Shifter

    As always, great investigative work!


    Not to belittle Bit Shifters accomplishments -- which are tremendous!! --, I couldn't help feeling a little bit curious how fast the 'simple loop' bench (Bench01) would run if BASIC10 had a sane implementation of integer variables and wouldn't have to do everything in floating point math.


    So I typed your program into my 2MHz BBC Micro that's resting beside my DevKit:



    I had to make a few adjustments since TIME is handled differently on the BBC, and I had to calculate the equivalent time a 3.5MHz BBC micro would need (line 120), so that we compare without difficulties.


    The results are more or less what I had suspected:



    that's almost 9 times the speed of optimized BASIC10 (and 18 times the speed of slow old Commodore Basic 10).


    Again: This is not criticizing Bit Shifter's work to make something usable out of BASIC10; my deepest respect for that! But I think it's a quite impressive demonstration of how much the MEGA65 would benefit from a (new) optimized implementation of BASIC with proper integer arithmetic (and some other missing things... ;-))

    I totally agree. Integer math would really give BASIC-10 a boost.

    Restricting BASIC to FP math was a decision of MS when ROM space was expensive and they had to cram the interpreter into 8KB.
    Later versions of MS-BASIC had integer math included, e.g. the interpreters for the computers with 6809 CPU (Thomson, Tandy CoCo).
    But Commodore didn't ever buy a new license, after 1977 and so all their interpreters are based on the old 1977 BASIC.
    The C= inhouse BASIC developers put more work into integrating DOS or graphics, but neglected speed optimisations.
    I could enhance BASIC-10 with integer math, but currently my focus is, to provide a stable as bugfree as possible interpreter,

    which should be ready for the release of the MEGA65 consumer version.
    Also FeralChild is already working on a complete new BASIC interpreter for the MEGA65, which is free from MS legacy code.

    Quote from Snoopy

    This resulting - I call it - "MEGA65 ROM" (because it doesn't work with the C65 emulator xc65 from LGB-Z anymore) is still in development and I'm glad that I was allowed to take a look at the current state 920116 already.

    I'm sure you got the point but of course it does not work in xc65 (that is, the Commodore 65 emulator in the Xemu project, not the MEGA65 one), as those ROMs uses MEGA65 features does not present on the Commodore 65.

    Quote from LGB-Z
    Quote from Snoopy

    This resulting - I call it - "MEGA65 ROM" (because it doesn't work with the C65 emulator xc65 from LGB-Z anymore) is still in development and I'm glad that I was allowed to take a look at the current state 920116 already.

    I'm sure you got the point but of course it does not work in xc65 (that is, the Commodore 65 emulator in the Xemu project, not the MEGA65 one), as those ROMs uses MEGA65 features does not present on the Commodore 65.

    Yes, I'm aware of that. That's the reason why I called it "MEGA65 ROM" (only my words). ;)


    It's the first ROM which couldn't be used in the C65 (xc65) because it is the intention to optimize it for the use of the MEGA65.


    So to say we are witnessing the split of the ROMs from C65 ones to MEGA65 ROMs. A very historical moment! :D

    Indeed. I'm still wondering (though I'm not involved in this work) if there will be willpower and time to do ROMs for C65s, ie even real C65 owners able to use that, which then fixes at least some bugs and unimplemented parts of the original C65 ROMs (surely without MEGA65 added advanced features not possible with C65 and this is about speed too, as many of the MEGA65 optimizations is to use MEGA65 specific CPU addressing mode which is much faster than doing MAP all the time, or issuing a DMA transfer just for a single byte [!] in some cases).

    Quote from Mac Bacon
    Quote from Bit Shifter

    The ELSE BEGIN ... BEND bug, that was in BASIC since BASIC7 of the C128 is fixed.

    Please explain - what exactly is buggy about that in Basic7?

    Code
    1. 100 IF (A=1) THEN PRINT "A=1":ELSE BEGIN
    2. 110 PRINT "A#1":BEND:PRINT "AFTER BEND"

    If you execute this code on a C128 or C65 and A=1 is true it prints "A=1" and skips the ELSE part.

    The bug is, that in this case the statement after BEND is skipped too.
    But the text "AFTER BEND" should be printed in any case, because it is not part of the IF THEN ELSE statement.

    Quote from Bit Shifter

    The bug is, that in this case the statement after BEND is skipped too.

    Of course it is skipped. BEGIN and BEND do not delimit a block (like braces do in languages with C-like syntax), they merely allow you to spread the effects of an IF over several lines. The "logical block" continues after BEND until the end of the line is reached in exactly the same way as it does without BEGIN/BEND:

    if x=1 then print "a":print "b" either prints nothing or it prints both a and b.


    I agree that this behaviour of BEGIN/BEND takes some getting used to, and it certainly looks strange if you're familiar with any language that knows about "real" logical blocks (like C) - but it is not a bug: it is defined this way, the C128 manual states how it works, and changing this behaviour will break a lot of Basic7 programs which expect it. I know because I have written some.

    Here is a good explanation of the BEND command:


    BEND


    Code
    1. Syntax:
    3. IF condition { GOTO lineNumTrue | THEN [ lineNumTrue | BEGIN ] } [ statementT ] [ : statementT ] ...
    4. [ statementT : ] ... BEND [ junk1 ] [ : ELSE [ lineNumFalse | BEGIN ] [ statementF ] [ : statementF ] ...
    5. [ statementF : ] ... BEND [ junk2 ] ]
    Quote

    The BEND at the end of the THEN clause (i.e., before an ELSE, if any) may be followed by any junk characters except a colon (:). This is because BEND is implemented using the code of DATA (although no values following BEND can be READ), in otherwords, BEND acts like REM, but another statment may appear on the line (REM does not allow that). Anyway, at the end of a THEN clause, BASIC will check what (if anything) follows the colon after BEND (it is looking for ELSE).


    The BEND at the end of an ELSE clause is effectively just like REM because BASIC will not search for another colon (:), but in general simply continue with the next line in the program. Thus junk2 may be any sequence of characters, in general. Note however that with nested IF/THEN/ELSE, BASIC may still be searching for a BEND after this one is found. In which case junk2 should not include the keywords BEGIN/BEND.

    Quote from Bit Shifter

    If programmers follow this guidance, they have no problems in new interpreters, where the bug is fixed.

    Correct, but that's a pretty fancy way of saying "Basic 7 programs written on the C128 might not work correctly" (even of they do not do any graphics stuff). :D

    Quote from Bit Shifter

    BTW, ubik detected another bug related to BEGIN ... BEND.
    If you nest these, the program will crash in some cases.

    IIRC the same recursive algo is used for finding LOOP statements, so that might be a problem as well. One could argue that a program with enough nesting levels to trash the stack is unreadable anyway, but fixing this should be easy: Just use a depth counter instead of recursion.

    Quote from Bit Shifter

    Yes, Snoopy , that's exactly, what I was referring to.

    Actually... no. The text referenced by Snoopy does not mention further statements after BEND at all, it only talks about "junk" as part of the BEND statement (like, before the next colon - what we talked about is what happens after the next colon).

    The C65 BASIC-10 is not compatible to the C128 BASIC-7.
    Keywords have changed, zero page addresses have changed, MMU is different, graphics library is completely different.

    BASIC-7 programs have to be manually ported in most cases.

    So there is no need to stay with the bugs from C128 in order to be compatible.


    And to be precise: I'm not workin on a C128 or a C65 BASIC.
    I'm working on BASIC for the MEGA65 only.

    This BASIC will not run on an orginal C65, but every MEGA65 owner is free either to use C65 ROMs or the new MEGA65 ROMs with the enhanced BASIC.
    So if there are any programs, that rely on bugs or quirks and cannot be ported for some reason, they may be run with the old ROM loaded.