Monday, September 10, 2018

Allow as low as 1 voice in CoolSoft VirtualMidi Synth

Get rid of CoolSoft VirtualMIDI Synth's hardcoded minimum of 16 voices limit.

So I was previously enjoying the CoolSoft VirtualMIDI ini file hack to allow lower than 16 voices to be set, however, in newest version such ability was completely removed by checking the limit in DLL file as well and overriding the ini set value (unlike previous versions). And I don't want to use outdated version just for this. So I went for an older DLL version which was 32-bit but kept overriding my beautiful voices value of 4, used IDA to show me some assembly, searched for voices and guessed the place (dword and 10h, which means 16, doh). Found it in hex editor and remembered the hex string of that assembly, the most important was 00 00 10 00 00 00, as with x64 assembly is a bit different. Downloaded latest version. But hey, there were a bit more than one 00 00 10 00 00 00, oh well... what's the maximum voices limit? 100 000, so now 100 000 in hexadecimal is? 01 86 A0! Now let's reverse it to have it little endian way. Now let's see if we get 00 00 10 00 00 00 and 00 00 A0 86 01 00 close to each other. Yup, of course there is such a place. And this place, in both EXE and DLL is for voices limit. Vua la, program fixed.

To allow as low as 1 voice in virtualmidi synth do the following:

For version 2.5.3 x64:
- get yourself a hex editor,
- make sure VirtualMIDI Synth is closed and no program is using MIDI (like chrome, steam, any music player) and that VirtualMIDI Synth program is closed,
- navigate to your DRIVE:\Program Files\VirtualMIDISynth,
- edit VirtualMIDISynth.exe in your favourite hex editor,
- change byte 10 [16] at position 3B33 [15 155] to 1 [1],
- navigate to DRIVE:\Windows\system32,
- edit VirtualMIDISynth.dll,
- change byte 10 [16] at position 2507A [151 674] to 1 [1],
- reopen VirtualMIDI Synth and change number of voices to 4 and use chiptune soundfont,
- ????
- enjoy 4 voice channels emulation.

Saturday, December 16, 2017

Teenage Engineering Pocket Operator PO-20 Arcade

Small, portable, mobile protracker-like synthesizer PO-20 Arcade with line-in, line-out (headphones), built-in speaker and display powered by 2 AAA batteries.

more demonstration videos at the bottom of this post

This little thing is amazing. It let's You create some cool beats even with Arcade version which is purely designed for chiptunes music. You have 2 octaves for most samples, some are 0-100 finetune range based instead - You control it with left knob. With right knob You alter the filter so the sound is more deep or fading into higher frequencies (kind of like flange effect). You can also do lead in real-time but I'm mostly programming all the patterns as it's very hard to rotate the knob just on the right note - this is unfortunately a con in this model that You can't use buttons in Note mode. Other models have this feature. You can program max 128-chain of patterns and chords which allows to achieve around 6-7 minutes of programmed music as long as You stick to low BPM like 60 and use 120 BPM beats. The max BPM is 240. A limit of 16 positions per pattern is a bit low but then You can use step multiplier which allows You to pick 2, 3, 4, 6 and 8 multiplier modes for a note witch can quite well overcome this limit. You can also easily copy a pattern so adding new instrument lines is very easy. Unfortunately You can't connect it to PC through USB port or similar to transfer Your creation in ProTracker-like form so if You want to keep the chiptune in mod format You would pretty much need to recreate it there. Nonetheless the device allows You to feel how the ProTracker-like music can be creater by showing You how to arrange your notes to play accordingly to given chord.

The device features:
  • a clock
  • an alarm
  • 2 octaves (A-1 to A-3)
  • 0-100 filter effect per note
  • 16 samples
  • 16 patterns
  • 16 chords
  • 16 live effects
  • 16 volume settings
  • 128-chains
  • parameter locking
  • a drone sound
  • fade out
  • tempo and swing
  • step multiplier
  • copying and clearing a pattern
  • various sync modes with left-right audio channel splitting
  • built-in speaker (I didn't know about that before buying! Nice feature!)
  • powered by 2 AAA batteries (mine rechargeable usually last 2-3 days)



Here are some examples of the music I was able to create by purely improvising:

House:
Rave:

Standard chiptune-like loop:

Techno Club:
Different bpm-swing variation of the above video, for demonstration purposes:
  • 60 bpm setting (120 real) / 80 swing
  • 65 bpm setting (130 real) / 50 swing

  • 90 bpm setting (180 real) / 50 swing


If you want to acquire this device (or a set of them) check out the product page: https://teenageengineering.com/products/po. They sure let you forget about the world around you when you step into music creation mode. :)

If you live in Poland or countries nearby, at this moment it's best to buy it from Czech distributor Kytary: https://kytary.pl/Search/?term=pocket%20operator. They offer reasonable prices and ship with DHL (select PPL as transportation method) and allow paying with Paypal.

Saturday, January 28, 2017

7-ZIP finetuner

Small application to try various 7-ZIP settings to find best combination for given input data. This usually results in smaller archive size.

Help:
7-ZIP Finetuner v1.00 by Mr_KrzYch00

This program will test various 7-ZIP switches known to improve
compression ratio. Smallest resulting file will be kept on disk.

Usage: 7zft [switches] [input files ...] [archive name]

Switches:
   /l#      - 7z executable (default: "c:\program files\7-zip\7z.exe")
   /d#      - dictionary size (default: 512m)
   /s#      - solid block size (default: on)
   /m#      - match finder cycles (default: 4294967295)

This program is provided as-is, use at Your own risk.

Support the author: https://www.paypal.me/MrKrzYch00


Optimization sample:
f:\MyScripts\7z test>7zft Ht_edf_v1 Ht_edf_v1
7-ZIP Finetuner v1.00 by Mr_KrzYch00

LC: 3, LP: 0, PB: 2, YX: 5, FB: 273 (INITIAL RUN) - best: 1372745
LC: 1, LP: 0, PB: 2, YX: 5, FB: 273 - best: 1372697  (-48)
LC: 2, LP: 0, PB: 2, YX: 5, FB: 273 - best: 1372483  (-214)
LC: 1, LP: 1, PB: 2, YX: 5, FB: 273 - best: 1372441  (-42)
LC: 0, LP: 2, PB: 2, YX: 5, FB: 273 - best: 1372264  (-177)
LC: 0, LP: 0, PB: 3, YX: 5, FB: 273 - best: 1372248  (-16)
LC: 1, LP: 0, PB: 3, YX: 5, FB: 273 - best: 1372061  (-187)
LC: 1, LP: 1, PB: 3, YX: 5, FB: 273 - best: 1371695  (-366)
LC: 0, LP: 1, PB: 4, YX: 5, FB: 273 - best: 1371445  (-250)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 273 - best: 1371404  (-41)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 272 - best: 1371393  (-11)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 269 - best: 1371342  (-51)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 260 - best: 1371305  (-37)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 253 - best: 1371303  (-2)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 252 - best: 1371115  (-188)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 172 - best: 1371084  (-31)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 155 - best: 1371069  (-15)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 141 - best: 1371010  (-59)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 134 - best: 1370992  (-18)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 125 - best: 1370980  (-12)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 121 - best: 1370964  (-16)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 70 - best: 1370908  (-56)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 68 - best: 1370846  (-62)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 63 - best: 1370812  (-34)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 54 - best: 1370781  (-31)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 52 - best: 1370756  (-25)
LC: 1, LP: 1, PB: 4, YX: 5, FB: 5              

    SUMMARY:
- Size: 1370756
- LC: 1
- LP: 1
- PB: 4
- Method: LZMA2
- YX: 5
- FB: 52




You can download it from: http://virtual.4my.eu

Tuesday, January 10, 2017

Zopfli on Odroid XU4

Lately I got myself Odroid XU4 microcomputer based on Exynoss 5422 octa core (4x 2Ghz + 4x 1.4Ghz). Surprisingly it can run all 8 cores simultaneously. Unfortunately its size is a bit bigger than Odroid U3 so if You made Yourself some custom made multi-odroid chassis, You would need to rebuild it if You plan to put XU4 in there as well. I mainly bought it to have headless OS with Zopfli KrzYmod running on, doing 999,999 iterations per block. I can confirm that it's around 50% faster than Odroid U3, with single threaded zopfli compression. In my small test of compressing a file using multi-threaded compression it was 55% faster than Odroid U3 O/Ced at 1.92Ghz (limited to 6 threads and various block sizes so it couldn't show its full potential, which I would believe could be up to 75% faster).



Unfortunately it's getting up to 95*C temperature when using all 4 big cores + 1-2 small cores, when small cores are not used the temperature goes to 93*C max. I'm now waiting for Grizzly Kryonaut thermal paste to arrive to apply it on all my RPis and Odroids (and later to apply better cooling solutions in case that won't help much) to make sure they won't get throttled much, especially during summer here.

There is also the same problem that occurs on Odroid U3 being SIGSEGVs with zopfli when long running certain blocks of data. The kernel seems newer on Odroid XU4, so I'm still confused by this error as it is not occurring with x86/x64 builds, might be GCC to generate bogus compilation, or one of the special switches I pass to it for faster builds.

Some tests:

                |       ORIGINAL -O2                   |  Mr_KrzYch00's Zopfli KrzYmod  |
       X        |-----------------------------------------------------------------------|
                | 2016.04.20 | 2016.05.19 | Makefile*  | v16.5.22 --t0 | v16.5.22 --t99 |
----------------------------------------------------------------------------------------|
           real | 15m35.201s | 14m6.836s  | 11m21.864s | 10m56.287s    | 7m11.551s      | - ARM Cortex-A9 - quad @ 1.92Ghz
Odroid U3  user | 14m40.965s | 14m6.480s  | 11m21.580s | 10m55.970s    | 19m30.025s     | (quad-core)
NEON       sys  | 0m53.925s  | 0m0.125s   | 0m0.095s   | 0m0.135s      | 0m0.805s       |
----------------------------------------------------------------------------------------|
           real |            |            |            | 7m21.517s     | 4m40.129s      | - ARM Cortex-A7 - quad @ 1.4Ghz
Odroid XU4 user | [missing]  | [missing]  | [missing]  | 7m21.345s     | 8m27.750s      | - ARM Cortex-A15 - quad @ 2.0Ghz
NEON+VFPV4 sys  |            |            |            | 0m0.080s      | 0m0.180s       | (octa-core)
----------------------------------------------------------------------------------------|
           real | 7m41.104s  | 5m50.010s  | 4m31.952s  | 4m1.706s      | 2m35.052s      | - Core i7-3630QM @ 2.7Ghz
Fedora x86 user | 7m20.247s  | 5m39.968s  | 4m24.474s  | 3m53.982s     | 6m27.671s      | (quad core)
rawhideAVX sys  | 0m9.306s   | 0m0.242s   | 0m0.071s   | 0m0.268s      | 0m2.843s       |
----------------------------------------------------------------------------------------|
           real | 5m59.811s  | 4m25.060s  | 3m38.561s  | 3m31.905s     | 2m17.897s      |
Fedora x64 user | 5m56.214s  | 4m24.860s  | 3m38.295s  | 3m31.776s     | 5m57.761s      |
rawhideAVX sys  | 0m3.543s   | 0m0.118s   | 0m0.208s   | 0m0.056s      | 0m1.636s       |


* - same original Zopfli 2016.05.19 with Zopfli KrzYmod's makefile + profile guided optimizations:
        https://github.com/MrKrzYch00/zopfli/blob/master/Makefile
  --t99 is to force all blocks compression at once - 6 threads used

Friday, December 2, 2016

Fedora rawhide kernel panic Virtualbox

I'm now running on a very slow PC being disassembled Asus 1215N with O/Ced Atom CPU to 2.35Ghz. As far as I managed to upgrade windows 7 to x64 version I can not make magically VT-x appear on this CPU, that was the reason why I ran into problems when tried to boot Fedora rawhide i686 in Virtualbox. Basically the problem occurs with kernels versioned 4.7+, it seems to boot fine with 4.6 one. On Virtualbox forums there was some discussion that it may be related to virtual box display driver from virtualbox extension pack being installed (and of course VT-x being unavailable making I/O APIC option not performing correctly without it - or so I believe after I did my tests). So far I have it running with kernel 4.6 and dnf-ing around the OS, that takes ages due to slowness caused by no VT-x being available. If I find a way on how to run kernel 4.9 with my configuration, I will share this information.

And here is some interesting stuff with DNF, it can even erase "None", lol:


Thursday, December 1, 2016

Chrome/Chromium hangs often

Recently I ran into a problem when my Chromium browser was hanging a lot when I activated the window or few seconds after opening new tab. First I VACUUM-ed every SQLite file in my profile directory... Didn't help... Then I run procmon tool to show me what exactly was it doing on my harddrive, then I found out it was reading some files, so I went through every one of them, renaming them by adding .bak extension. And then vua la, I found out what was causing it. It turns out it was Preferences file growing too big (even though it was just 450KB) when You visited a lot of various sites in the past, as it saves some preferences for every one of them and the engine/parts of code to read it back and analyze seem to be very inefficient, not to say, shitty. So imo that part requires some kind of inspection to be performed by Chromium/Chrome DEVs as currently it's giving Chromium even 1 minute hangs every now and then.

To fix Chrome/Chromium hangs:
- close all running instances of Chrome/Chromium,
- go to Your Chrome/Chromium Default profile, usually %LocalAppData%\Chromium\User Data\Default for Windows users,
- find Preferences file,
- rename it to Preferences.bak or remove it,
- run Chrome/Chromium,
- lags/hangs should be gone now,
- in case they are not, You may be having some other problematic file that, maybe, uses JSON format to store data as well. In other words its structure should be having a lot of { } " " : chars as well, maybe the other one got too big for You?

You may have some settings reset to default like Chrome/Chromium asking to be default browser or having Translate messages scaring You a lot, unfortunately, You must get past it again now. :)

Monday, October 3, 2016

Analysis and conversion of binary files with the use of PHP CLI

Last time I gave myself a mission to translate some game to Polish language. These aren't ordinary text files but instead custom scripts binary coded. At the bottom of file there is a lot of letters in a row. You can't practically tell which part of the text will be displayed in the exact moment in the game. By deep analysis of such files You can come to some conclusions. By viewing one of them in hexadecimal editor You can notice that some characters repeat over and over again. You can notice that they occur every 4 bytes, so these are most likely binary coded 32-bit variables (long int). However, as it is a script that hold a lot of numbers, then You can imply, that the big part of them will be of no use for You. Also by carefully looking at it You can notice repeating [FF] bytes one next to another. So these will be most likely numeric variables represented as signed type (positive & negative), because I doubt that the software the company used to edit these scripts required anyone to type 4 billion-like numbers, these were most likely negative numbers close to zero like -1 or -3.

Why would You need to know all of this though?
Such detailed knowledge about these files is needed to have full control over the length of text You wish to modify. As I previously wrote, such text doesn't have a visible split points, in which part of the game which part of the text is shown. So something else must used for game to tell the beginning at which to start loading the data and it's length.
Let's start from the beginning of file. You can see a DCPB tag there which identifies file type used by game. Next You can try to decode 4 bytes after that, because it may be used to represent an offset or certain data length in this file. So [BC] [BA] [01] [00] is: 188 + 186 * 256 + 1 * 256^2 + 0 * 256^3 (little endian) = 113 340. Now You can check the file size, it is 117 150 bytes. So exactly how I wrote, it must represent a length of some data in this file. So now You can check if something interesting is located at the decimal address of 113 348 in this file. [64] [00] [00] [00] is located there, in other words the value of 100. But what it actually means? An offset? A value? Something else? If this is an offset then for sure at the distance of +/- 100 bytes nothing interesting exists. Let's go to 4 bytes ahead then. [20] [03] [00] [00] is located there, in other words the value of 800. Let's see if something interesting exists 800 bytes ahead of this place. Bull's eye! By adding 800 to current offset, from which we read this value, first text character exists. So You can be almost 100% sure that this is the offset of first part of the text which will be displayed somewhere in the game. Now You can go 800 bytes back and read next 4 bytes as signed long int. [12] [00] [00] [00] is located there, the value of 18. Is it maybe 800 + 18 as a some kind of offset? Hmm. Let's check another 4 bytes ahead, [32] [03] [00] [00], the value of 818. It looks like that in exactly THIS PLACE the offset really is coded. So what exactly means the value of 18 we previously decoded? 800 + 18 = 818! If You assume that 800 and 818 were offsets, then what occurs after ever of them (in this case the value of 18) is most likely the length (the number of bytes) of the text that will be read by the game. Now You can count how many occurrences of such 8 byte ranges exist before You reach the place where the text resides. It looks like there is 100 occurrences of them, so 4 bytes coded before the first offset ([64] [00] [00] [00] = 100) most likely indicates the amount of lines of the text to translate.

Let's begin?
After getting more or less accustomed to file structure, which You are interested in, You can translating the text. But let's think for a bit. If You have 100 occurrences, then every of them needs to have offset and length properly aligned. Won't there be a problem if the first part of the text gets a bit longer than the original? It is known that other languages than the original, the game was transcribed in, may result in bigger amount of translation data. You would then need to manually alter every next offset! Huge amount of work and a lot of time required. Is there maybe some other way to do this?

PHP CLI scripts as a helpful tool to convert files to more user friendly version.
Easiest way is to "convert" the file in such a way that You will have the text exactly split line by line, You won't be bothered by some kind of offsets, and You can skip directly to translating the text. You must remember, though, that such file must be properly converted back to its previous format, including the changes to binary data that points the offsets and lengths of parts of the text/lines. You can attain much more when programming in C, for example by directly reading the file and dynamically creating text input boxes for text inputting/altering basing on original game script. However, it can take more time to do it, it all depends if You have parts of code to use from Your other projects, or not.

Analysis and conversion of files with the use of PHP scripts.
You can write PHP scripts by checking the manual and using plain notepad. First You would need to create an algorithm or have the ability to do it on-fly in Your head. You must have in mind the analysis You did before. Let's try to more or less describe how the file structure looks like:
String: "DCPB",
4 bytes of signed long int: data length, which we You won't modify,
data, which You won't modify,
4 bytes of signed long int: amount of text occurrences,
{4 bytes of signed long int: nth text offset, 4 bytes of signed long int: nth text length} * amount of occurrences,
text as a whole.

By looking at the above You would for sure need a loop/function to read 4 bytes and translate them to long int. For sure You will also need to read a parts of data which You won't modify, and as such, they would need to be coded in a way that it will be safe to text edit such file. I advice to use base64_encode, and to apply deflate compression before it:
base64_encode(gzcompress($buf,9));
This way You will achieve a file to edit which will be much smaller that the original and it won't hold a lot of numbers and letters You are not interested in at the beinning (base64).
When You will be in a place to read offsets and lengths of the text, then for sure You will need dynamic arrays, which will hold these offsets and lengths in 2 arrays (or 2-dimensional array under one variable, which one You prefer most). In PHP it's relatively easy, in C You would need to malloc/realloc a lot, and to remember to NULL byte at the end of array of characters etc. Dynamic arrays are best created here by loop repeating [text occurence amount] times by reading 4 bytes two times, decoding them and putting them as long ints in 2 array variables, for example:
for($x=0;$x<$amount;++$x) {
  $buf = fread($file1,4);
  $pos[] = (ord($buf[3]) << 24) + (ord($buf[2]) << 16) + (ord($buf[1]) << 8) + ord($buf[0]) + 12 + $rawdatalength;
  $buf = fread($file1,4);
  $len[] = (ord($buf[3]) << 24) + (ord($buf[2]) << 16) + (ord($buf[1]) << 8) + ord($buf[0]);
}
Of course You can read whole 4 bytes and use proper bitshifting or other methods. Whichever method suits You best, most important is to do have conversion working without errors. Code performance in usually required in production environment or when a lot of data is concerned.
After the loop finishes You need to start another one, which will use data from previously created arrays, properly read the amount of characters and output it in separate lines into text editing friendly file. So to be sure it's best that You use fseek for offsets and fread to read amount of characters, and as last fwrite. Example:
for($x=0;$x<$amount;++$x) {
  fseek($file1,$pos[$x],SEEK_SET);
  $text = fread($file1,$len[$x]);
  fwrite($file2,$text . PHP_EOL);
}
This is more or less everything You would need to do to create user friendly text editing file. You can of course put an amount of data You got from compressing and base64-coding the data. This way You will avoid the fgets error of 1024 characters limit if You will utilize fgets to read Your text files. In case of the data that You will base64 decode and decompress, You will use fread($file,$amount_of_bytes_to_read).

The converting back to binary script needs to be done in reverse order. For sure You would need to recreate "DCPB" tag and write 4 bytes that You will read from the length of Your decoded, decompressed data, write this data, recreate proper offset and lengths of lines which Your script will read from text file You used to safely translate the game text. After You finish writing the coding part You need to test the original coded file against Your translated one, if the differences between the two start after first offset pointing to the text and if all offsets and lengths properly point to the parts of translated text. If You can confirm that it's exactly as it should be then You succeeded in creating coder/decoder of script files which will make Your translation alteration much easier.

File used as an example: http://virtual.4my.eu/samples/SSS05_01.scripb