User talk:Malvineous

Wolfenstein 3D -> Wolfenstein 3-D

I've corrected the infamous 3D tyop in the names of several Wolf3D-related pages; can you please delete the following ones?

Category:Wolfenstein 3D -> Superseded by Category:Wolfenstein 3-D
Image:Wolfenstein_3D.png -> Superseded by Image:Wolfenstein_3-D.png

Thanks :) --De Zeurkous (zeurkous@zeurcomp.nichten.info), Thu Nov 22 08:18:42 UTC 2007

Haha, you've certainly been busy. Don't forget that this wiki isn't meant to be an encyclopaedia of the games, its main purpose is documenting the file formats and editing tools that can be used with the games, as opposed to how the games behave without any mods, or what they might be called if you forget to change the title screen :-) -- Malvineous 11:14, 22 November 2007 (GMT)

Nah, I haven't forgotten. As we're reinventing the wheel for a semi-virgin wiki, the naming and something static like the cheats seemed like a good place to start :) --De Zeurkous (zeurkous@nichten.info), Thu Nov 22 12:20:04 UTC 2007

Jill of the Jungle palette

Hi, I'm currently working on a level viewer for Jill of the Jungle (maybe an editor later on). Some of the information I found here has been very useful, but there's one thing I'm a bit stuck on. On the SHA Format page it says:

Each array item contains 4 unsigned bytes, representing CGA, EGA and VGA, respectivley, and then a null value. [...] Each byte maps to the index of a colour in the palette. If there are no colours required, the tile set uses the default colours.

However, it doesn't say anywhere where the palette can be found. Is it stored in the EXE file? I hope you can help me on this :)

Regards, Spinal 13:00, 2 March 2008 (GMT)

Unfortunately I didn't write the JotJ SHA page, and I haven't yet had the time to figure this out. For Xargon I've taken the quick and dirty route and used a screenshot from within DOSBox as the source of the palette, until I can figure out where the palette is located. If you find this out, don't forget to update the page! :-) -- Malvineous 22:02, 2 March 2008 (GMT)

Hey Spinal, sorry, I wrote the SHA Format page, so that was probably my bad. I wasn't able to find the palette either. I assume it's in the .EXE, however I spent a fair bit of time manually mapping it using a Wombat tool (http://www.szevvy.com/node/3). I was also writing a level viewer (initally in XHTML which turned out to be too slow, so then I moved to Flash 9). I tried to email you my findings, but the email bounced. I'm not sure if they were 100% accurat, which is why I never added them to the documentation.

Hey Malvineous, it seems the Jill - DMA Format page has been removed. Do you have a log of what happened to it? I'm sure it existed at some point.

- Daniel 20:00, 5 March 2008

I don't ever remember seeing a DMA page - certainly there's nothing in the wiki logs about it being deleted or renamed. Maybe you started working on it but never got around to saving it? Or is it one of the Xargon pages that shares the same format as Jill? (by the way, you can type ~~~~ to automatically put in the signature in these messages.) -- Malvineous 11:28, 5 March 2008 (GMT)

Sorry, it must have been the stuff in the SHA Format page that I was thinking of. I may have added the link to the DMA Format page, and never expanded on it. - Daniel 02:46, 6 March 2008 (GMT)

Hi guys. Sorry for the mistake, I mixed up the authors of the SHA and level formats. I've written a level viewer in PHP the last week, by using static images instead of getting them from the actual files. This causes some problems for episode 2, which uses a different palette, but besides that it works great :) When everything is done I'll upload the maps to my server and the tile/object lists to this wiki.

@Daniel: strange that it bounced, I confirmed my address so it should be working fine... You can mail it to spinal -at- zanderz -dot- net. Also, I saw on your Wombat page that you "have a vague idea of how to do" Hocus Pocus. Could you share your findings on this wiki? I'd love to make maps, or even a level viewer/editor for that game :) -- Spinal 00:32, 8 March 2008 (GMT)

Okay, with help of Daniel who send me the palette I found it! I'll put my findings on Jill of the Jungle palette. -- Spinal 15:36, 13 March 2008 (GMT)

Halloween Harry/Alien Carnage

Hi Malvineous. Thanks for your edits on the GMF article! I notice you mentioned there exist versions of the game which don't require exploding a BNK to get at the game data. What does the directory structure look like for the version you have? I'm working on a map viewer (which will become a map editor at some point) and would like it to be able to work on whatever versions are out there. Thanks for any info. -Duckthing 04:41, 11 December 2010 (GMT)

Hi Duckthing - no worries, Halloween Harry has long been an interest of mine, but my map editor sadly needs a lot more work before I can begin reverse engineering more file formats so I'm glad you've made a start! AFAIK technically Halloween Harry has no compression, but for whatever reason (probably listed in the Apogee FAQ) they decided to swap episodes 1 and 3, compress the BNK file and then re-release it under the name Alien Carnage. You can download all available versions (including registered versions since the game was released as freeware in 2007) from RGB Classic Games. BTW not sure if you noticed but I also posted a couple of things on the GMF article's talk/discussion page. -- Malvineous 06:08, 11 December 2010 (GMT)

Thank you! Working with the uncompressed version will keep things less complicated. I did notice the GMF discussion you added, but haven't had a chance to really go through it yet. -- Duckthing 06:41, 11 December 2010 (GMT)

No worries. If you're running Linux or MacOSX you should be able to compile libgamearchive which will let you copy files in and out of the BNK files. This is a library so it's also possible to use it in your own program for accessing and editing the map files while they're still inside the BNK, for instance. (BTW the 'minor' tickbox should only be used when the edit does not affect the information on the page - e.g. fixing a typo or a broken URL. If you are changing the actual content on a page it is never a minor edit!) -- Malvineous 07:51, 11 December 2010 (GMT)

Dangerous Dave

Working on this, does anyone have any info on the formats? The EGA graphics are 84KB and external, but CGA and VGA graphics unrelated to them are in the 174KB executable, apparently uncompressed, I don't know how this is possible. I've found the levels and partially reverse engineered the format (The first 1000 bytes, a 100x10 array of tiles.) but each level is followed by ~280 bytes of variable and unknown data I assume is sprites. I have a list of internal files in the executable, looks like most of it is non-game code. Any help would be appreciated. -Endian? What are you on about? 06:03, 12 December 2010 (GMT)

Hmm, interesting. All I can suggest for the levels is - as you're probably already familiar with - changing the values in a hex editor then loading the game to see what happens. I would suspect that all the graphics would be in a similar format, so if there are any headers or structures used in the EGA graphics files perhaps they're duplicated for CGA/VGA? Also John Romero seems pretty accessible, perhaps you could ask him directly? Failing that, randomly writing data into the .exe until the images change could be used as a last resort! -- Malvineous 12:09, 12 December 2010 (GMT)

Ha, that was easier than I expected - uncompressed VGA data is easy to find with a good hex editor ;-) This looks like it -- Malvineous 12:17, 12 December 2010 (GMT)

Ok that was only the menu graphics. At offset 0xC629 there's some sort of offset table which looks very similar to the contents of EGADAVE.DAV but there are extra bytes dotted all over the place - perhaps it's RLE encoded? Actually just before that there's the number 0x6CAA. If you go a bit less than 0x6CAA bytes after this you end up with another table, which to me suggests that 0x6CAA is the decompressed size of that first table. -- Malvineous 13:01, 12 December 2010 (GMT)

If I remember it correctly, the VGA sprites and tiles are RLE compressed and it's not too hard to decompress them. The positions of the enemies in the levels on the other hand are still a mystery to me. Calvero 20:43, 12 December 2010 (GMT)

Looking at the levels, which directly follow the VGA palette, they seem to contain 256 bytes of header data, then the 100x10 tiles, then eight bytes of lead-out data. Then the next level's 256 bytes of header, 100x10 tiles, then eight bytes of lead-out. Although I say "level", the first "level" seems to contain both level 1 and 2, and this seems to be the way it works in the game - in some levels you can see through to others. I'm not sure what the 256 bytes of header data are for, but some levels have FF FF 00 00 repeated which seems to indicate two uint16 fields which are blanked out for those levels. -- Malvineous 13:15, 13 December 2010 (GMT)

First up, levels; where does level 1 start? It's possible I'm reading the start of a level as the previous level's end, I looked for the start of tile data, which I assumed, like Keen 1, would be the level start.

Next, palette; I'm assuming, as you said, it is located just before the level data?

There is VGA and CGA graphics in the executable, plus headers, but the bulk of graphics for CGA and VGA I haven't been able to locate. If the data is RLE compressed I should think it would use either Keen 1-3 fullscreen or Jazz Jackrabbit tile RLE, both of which I know well. I'll look into both of these options, cheers! -Endian? What are you on about? 12:03, 20 December 2010 (GMT)

I've found the VGA and CGA data for the menu graphics. It looks like the CGA data is stored in a series of 1-bit-per-pixel (monochrome) images. Or it could be EGA data with the planes split out (e.g. red plane for all images, then green plane for all images, etc.) I think I found three planes which means it's probably EGA, as CGA would have a max of two planes. I've also found the font graphics, this looks like 4bpp/16 colour EGA data. Haven't seen level graphics yet. I've successfully modded the levels with a hex editor (putting the jetpack everywhere) so now I've finally looked at all the levels! The creatures are definitely not stored in the 100x10 tile data. Putting random bytes in there revealed that the values are indices into the image files, because if you put the value in for fire it will animate, and the following three or so values show up as still frames from the fire animation.

The VGA palette starts at 0x26B0A (I updated the main DDave page with this) + 768 bytes leads straight onto the start of level 1 at 0x26E0A. So at 0x26E0A there are 256 bytes of data, then 100x10 tiles, then 24 bytes of footer data. Some of this data (perhaps footer data) might be the level start coordinates, because Dave doesn't always start the level at the same place (e.g. the warp level when you climb the tree on level 5 and jump off the edge of the screen.) -- Malvineous 12:24, 20 December 2010 (GMT)

I think I've found all the files so far, I've updated Dangerous Dave with the offsets of the files, including what appears to be CGA and VGA tiles. I'm not sure what sort of format they're in though, it looks like they might be compressed (and so the offsets in that list might be off by ~4, as I may have cut off the decompressed-size field.) I've also updated the map format page, I think it's pretty much fully reverse engineered now. -- Malvineous 05:13, 21 December 2010 (GMT)

Noahs Ark 3D and IMFs

Been poking around Noah's Ark 3D, an adaption of the Wolfenstein game and found a few interesting things which I'm hoping to confirm.

First, the game uses MID files, not IMF files for music, second, the MIDI files have an extra word at the start of the file giving the file length. Third, the code uses the first word of the midi file as the length of data to read from the file and put into memory to play. After poking around a bit in Wolfenstein and Keen I believe this is the same with IMF files, that is, ALL IMF files are type 0 as it were and the type 1s are read just like type 0s by the games (The first word is read as the amount of IMF data to copy into memory to play, this is then read as type 0.) I'm not quite sure what this does for the two types since type 1 is would essentially be a type 0 with a header and footer that are part of the AUDIO.xxx format, much like PC\adlib sounds can have the huffman lengths at the start. Are there any games that use 'individual' type 1 IMFs (Even if included as part of a group file. All I've seen are games like cosmo, where non-AUDIO IMFs are type 0.)

Should there be a Noah's Ark 3D page and should we add the midi note to the AUDIO format page? It seems that the format can hold any type of music file the game can read, not just IMFs. -- Levellass

Yes it is known that the type-1 header might really be part of the AUDIO.xxx format but alas there were many type-1 files floating around the place (incorrectly extracted, you could argue) before this was realised, so the format variant is here to stay. It does have some advantages though (like tagging files with the song title) but I'm not sure whether anyone has figured out definitively whether the header (and footer, in the case of the Wolf3D files) belongs to the IMF or the AUDIO file. I guess you'd have to browse the game source code and see how the game uses the data to know for sure.

As far as the MIDI goes, I believe the AUDIO.xxx file is just a container file like any other, capable of supporting any data at all. It's up to the game to decide how to use it. If anything this makes me think it would fit better if the AUDIO.xxx article was merely documenting the structure, with links to actual audio formats like PC SFX, Adlib SFX, IMF and MIDI and notes about which games make use of which formats.

Oh and yes, by all means create a page for the new game. -- Malvineous 08:41, 3 February 2011 (GMT)

Sound Files category

I don’t know why there’s a Sound Files category and also an Audio Files category... it seems redundant. I left those categories the way they are though.

Also, the File format data types page needs some signed ints, but I didn’t know what naming convention you wanted to use. I described something as an INT16LE on BMC Format. Yellowantphil 20:12, 11 September 2011 (GMT)

Not sure about the Sound vs Audio - I use the term 'audio' to mean both sound and music, but you're right, if there are separate sound and music categories we don't really need an audio one as well. Feel free to update. INT16LE is perfectly fine too! I added a note to the data types page. -- Malvineous 22:30, 11 September 2011 (GMT)

GamePage template

Should the GamePage template add the page to its own category? It seems a little odd that Wacky Wheels is not in the Wacky Wheels category. Yellowantphil 04:51, 16 September 2011 (GMT)

I decided against this because there is a link to the game's page in the category intro at the top of the page. It seemed unnecessary having another link in amongst all the other pages. This was also because the game page was really only meant to be a summary, there wasn't supposed to be a huge amount of detail on it, only links to other pages with the detail (which then would appear in the category list.) I would imagine that, eventually, Wacky Wheels would be tweaked to move a lot of the content onto subpages. -- Malvineous 12:58, 16 September 2011 (GMT)

Scope of the wiki

I don’t know how extensive you want the information to be on ModdingWiki... I added a few lists of audio and graphics files a while ago, and there will probably be other information that I’ll find out that’s potentially helpful for level editing but not directly related to it. I can move some information to my website if it starts to become out of scope for your wiki. I also sometimes end up documenting everything imaginable... there’s some minutiae on my website and possibly on your wiki that I ought to go through and delete. Yellowantphil 05:28, 10 October 2011 (GMT)

Hmm, that's a difficult one - I'm always in favour of documenting more rather than less. Generally speaking if it's related to a game and it would help either a programmer writing a utility or a someone trying to change a game manually then it should be documented. About the only thing that comes to mind that's over documenting is when some small calculation is expressed in a couple of different programming languages, but I've left that in so people can just copy it rather than trying to figure it out from the text. But the Wacky Wheels page is probably the gold standard for what all wiki pages should look like :-) Do you have anything in particular you're unsure of whether it would fit here? -- Malvineous 10:07, 10 October 2011 (GMT)

I was mainly thinking of the list of VOC files on Wacky Wheels Music and Sound Effects and the list of *.sp files on Wacky Wheels Graphics Formats. I have an expanded list of the *.sp files on my computer but it isn’t finished yet. I guess there isn’t all that much more Wacky Wheels information to be added to this wiki though. There is still some modding information on my website that ought to be moved over here, so that would take up another page or two. I’ve been lazy about doing Wacky Wheels stuff lately, but I might find some time for it this weekend.

By the way, I guess that guy from Cascadia Games emailed you. It’s pretty interesting that WW might become an iPhone app. He told me that he was going to be using his own game engine, which sounds a lot easier than trying to get a hold of the WW source code and porting it. —Yellowantphil 16:46, 15 October 2011 (GMT)

I was going to say the list of VOCs might be overkill, but then the list of .sp files might be useful. To be honest, it's the sort of thing that you could probably omit if you were feeling lazy, but likewise I don't mind at all if you choose to include it, so since it's already there I'm happy to keep it.

Yes I did get an e-mail from Cascadia, I was going to pass along your details but it sounds like you're already aware of it! Using their own engine might make it a bit smoother than the original, as the jerkiness of turning is my only criticism of the game. -- Malvineous 00:14, 16 October 2011 (GMT)

I'd like to document not only the format of some game archives but also their contents. This would be quite long lists of basically images, sprites and some minor stuff. I'd also like to upload the actual images alongside but I'm not sure whether that fits the scope of the wiki or if I should just put it somewhere on github. Any advice highly welcome :) -- Yetmorecode 15:25, 22 December 2020

That depends on what you mean with "their contents". Documenting the used file formats is obviously in the scope of this wiki, since there's not much point in being able to extract/replace the files in those archives if you can't actually view those files or make those replacements. But dumping the whole list of all existing files in a game into a wiki article is usually not very useful. An article could highlight some specific files with special functions in the game (stuff like the remap-defining PALETTE.CPS in Red Alert 1 comes to mind; cps is a very normal image format in these games, but that specific file controls all recolouring of sprites that change to their owner's colour), but a list of all sprites and sound files and all their exact usages in the game doesn't seem like it'd belong here; most of the time such things are pretty self-evident.

PS: when you add something to a user talk page, use the signature code to give your note a clearly visible author and date stamp. There's a big red box on the top of the edit page telling you how to do that.

--Nyerguds (talk) 15:51, 22 December 2020 (UTC)

Alright that makes sense. Sorry about the signature I must have been blind. Yetmorecode (talk) 17:11, 22 December 2020 (UTC)

Yes just to confirm this, an image or two is useful to illustrate a file format but we don't want to become an archive of all game material, not least of which because extracting this is often against the game's licence (and doesn't really come under "fair use") so could open you up to legal trouble if the current copyright owner takes exception to it. If you come across something hidden or undocumented you can put it on The Cutting Room Floor otherwise GitHub Pages may work as you've suggested (if you fly under the copyright radar!) -- Malvineous (talk) 07:08, 23 December 2020 (UTC)

Wacky Wheels

Basically all of the WW modding information from my website is here now. I moved one page when I realized it should probably be capitalized to match the rest of the pages on this site... you can delete the redirect at Wacky Wheels high score files if you want to.

Now, I need to remove most of the modding information from my website and refer people here. Then we’ll have reached the glorious future where all WW modding information is in one place. I’m going to keep those two text files of notes on my website though—the information in them is disorganized and I think not 100% accurate, but they might have some information that isn’t here yet.

By the way, how’s Camoto going? I’m planning to use it to make a new WW level, but I won’t have time for it for a few more months. —Yellowantphil

Sounds good! Thanks for all the effort you've put in by moving it across. I think Wacky Wheels is now the most completely documented game on the wiki! Also can't hurt leaving your own info up - sometimes it's nice to go back to the original source if something is unclear. Camoto's going well, but unfortunately I haven't had much time to work on it over the last few weeks. You probably won't be able to make much of a level until I add in the ability to select a tile from the tileset, but that's in progress and hopefully will be ready by the time you are! Keep me posted when you do start your level(s?) -- Malvineous 12:22, 2 February 2012 (GMT)

Well, as I was reading some of the information on my website, I noticed that it isn’t written very well. I wrote those pages back when I had very little experience programming. Anyway, I’m planning to redesign my website soon, and this will save me from having to rewrite a few of those pages.

The tile editor in Camoto as it is now is still certainly useful—I can add new tile types to a level using a binary editor and then move them around in Camoto. I think that it shouldn’t be too hard to make a new WW level using the Gimp and Camoto plus some binary editing. I recall having some problems with the route editor for the computer players, but I didn’t look into it at the time—I’ll let you know when I’m working with Camoto again, and I’ll try to come up with some useful bug reports or fixes if I run into problems. —Yellowantphil 14:06, 2 February 2012 (GMT)

Catacomb

Hey Malvineous, do you know anyone who has properly read the Inverse Sound Frequency file format and got the sound playing? I'm struggling with some fine details in my game. Sounds play near what they should sound like, but the sound quality is all crackly and some sounds play totally incorrect. I followed a basic code sample by Anders Gavare. Does your tool support this? Interested in helping me figure this out? My current code is available at catacomb_sound.c - Eros 11:57, 5 May 2012 (GMT)

Should this be moved to my talk page? - Eros 11:59, 5 May 2012 (GMT)

Probably best to keep it on my talk page if you want me to respond in a timely manner :-) I based my old utility off Anders' one too, but it's been a long time since I looked at the code. My latest utility doesn't support sound effects yet. I know Levellass has written a few utilities to play and edit those sound files, so I think your best bet would probably be to post over at the Keen:Modding forum. Do note that PC Speaker effects do sound crackly at the best of times, though! Also make sure you've got the right audio format (8-bit vs 16-bit, signed vs unsigned) - I remember some inconsistencies there causing some very distorted results for me. I love the name 'Cataclone' - does your code compile under Linux? -- Malvineous 13:21, 5 May 2012 (GMT)

I currently compile the code using Code::Blocks with GCC 4.6 -std=gnu99. It *should* compile under Linux. But I haven't tested that out yet. Catacomb sounds also have flags attached to each one, It's not exactly the same specification as what it says @ the Inverse Sound Frequency format page. Perhaps the flags would make a difference to the sounds? I have the sound setup @ sound_manager.c in the sound_manager_init() method. I use AUDIO_S16 (signed 16-bit) @ 22050 Hz with 2 channels. I have not added support for the priority of the sounds yet, I plan on doing that after I get them playing properly.

I found some information on the ROTT sound format here: _rt_soun.h. This format should be similar to Catacomb as ROTT was developed by Apogee using the Doom engine. Here are the flags for the sounds that they define:

#define SD_OVERWRITE     0x01
#define SD_WRITE         0x02
#define SD_LOOP          0x04
#define SD_PITCHSHIFTOFF 0x08
#define SD_PLAYONCE      0x10

There are also many other sound files for ROTT: [1], [2], [3], [4], [5]

I tried it last night and I was able to get it to compile (after a few changes, like using '/' instead of '\' in a couple of #includes) but all I got was a black window and a beep sound, then nothing. How far advanced is the project? Should I expect more than that? I would expect small variations in the file formats between games, but I would be surprised if the core sound data has changed over time. -- Malvineous 02:38, 6 May 2012 (GMT)

On Windows, you can run around the levels, collide with walls, collect items, etc... I still need to implement monsters, AI and power shots. However, that beep means that the sounds playing. (see line 71 in main.c: catacomb_sounds_play("foundsound");). I just committed the catacomb_data.h file. Update it accordingly, you should now see much more than a black screen.

Oh wow, that works really well now! I see what you mean by the sounds. You might have to output the PCM data to a .wav/.raw file and look at it in a sound editor to see what's going on. It sounds like there might be wrapping or something going on. Also some sounds (like being blocked by a door) play multiple times over the top of themselves. I guess this is what you meant by priority not being implemented (although maybe you wouldn't need to handle priority if you can play multiple sounds at the same time, unlike the real PC Speaker.) One suggestion - if you do decide to support Linux, abstract the calls that open files so that they search for filenames instead - that way you don't have to rename all the Catacomb files to be uppercase, since Linux has a case-sensitive filesystem and most filenames are usually all lowercase. -- Malvineous 03:42, 6 May 2012 (GMT)

I have exported the raw data to a file, then imported the raw data into Audacity using various settings. All of them sound like they do in the game, crackly and sometimes distorted. I really think its some sort of post FX used by the flags. As for files, I'll code some sort of SafeFileOpen, SafeFileClose functions like the id codebase that open files regardless of case when i get around to implementing the Linux version. - Eros 03:54, 6 May 2012 (GMT)

If you can e-mail me one of the files (preferably a 'quiet' one that's not supposed to be crackly) I can take a look if you like - malvineous@shikadi.net -- Malvineous 03:57, 6 May 2012 (GMT)

Merge

Is there a way to tag two pages such that it is clear that they should be merged? For example, Crystal Caves Tileset Format describes the same format as Duke 1 Tileset Graphics.Frenkel (talk) 21:00, 15 July 2013 (GMT)

You can mention it on the talk page for one of the articles. Any suggestions as to what we should call the merged page? -- Malvineous (talk) 03:09, 20 July 2013 (GMT)

Dark Ages is the first game that got released that uses this format, so Dark Ages Tileset Format could be the name, like all those formats that are named after Commander Keen 1-3.

Or we name it after Duke Nukem 1, because that game is probably more popular than Dark Ages, Crystal Caves and Secret Agent, like Commander Keen 1-3 Level format, although Slordax used it before Commander Keen 1-3 did.Frenkel (talk) 19:13, 20 July 2013 (GMT)

Yes I was thinking along the same lines, naming it after the first game that used the format. But then the release date doesn't necessarily indicate which game "invented" the format, if one game took longer to develop and was released late. I was wondering whether there is a common thread amongst all these games, such as the same programmer or company we could use instead. In many ways I would prefer a name like "Replogle tileset format" or "Apogee EGA tileset format". What do you think? -- Malvineous (talk) 23:36, 20 July 2013 (GMT)

I've read the | Apogee Legacy interview with Peder Jungck again today. He wrote the ProGraphx Toolbox.

DN1.EXE contains the string Version 1.0 EGA/VGAProGraphx EGA/VGA Toolbox,

CC1.EXE contains the string Version 1.5 320x200 PAN EGA/VGAProGraphx EGA/VGA Toolbox,

DA3.EXE contains the string Version 1.0 EGA/VGAProGraphx EGA/VGA Toolbox (DA1.EXE and DA2.exe contain gibberish).

So I would suggest ProGraphx Toolbox tileset format. Peder also wrote a book about the toolbox called Graphics Programming & Animation: Ultra-Fast Assembly Routines for EGA/VGA Graphics Animation. Maybe that book has a better name for the file format. Frenkel (talk) 11:03, 21 July 2013 (GMT)

So we know of three different versions of the ProGraphx Toolbox. Dark Ages and Duke Nukem 1 use version 1.0. Each tileset is 8064 bytes long and is in a separate file. Crystal Caves uses version 1.5 and each tileset is 8003 bytes long, but they are all in the same file. Secret Agent uses version 2.0 Each tileset is again 8064 bytes long, they are all in the same file and the file is encrypted. Frenkel (talk) 17:00, 28 July 2013 (GMT)

Thanks for figuring all that out. I've merged the pages and changed the name to ProGraphx Toolbox tileset format as suggested. Please feel free to correct anything I have missed. -- Malvineous (talk) 02:40, 4 August 2013 (GMT)

Guidelines for proper Wiki Editing

Dear Malvineous! Could you post the guidelines (or guide) for proper editing of the Wiki? The coder (talk)

Hmm, there isn't really one, it's more about finding a similar page and matching the style found there! Copy and paste works well :-) There is ModdingWiki:Contributing but it's not very detailed. (BTW please put ~~~~ at the end of your messages, when you preview the page you will see this gets replaced with your name and the date automatically.) -- Malvineous (talk) 21:59, 12 December 2014 (UTC)

VGFM Music Format

Hello, Malvineous!

I accidentally found that VGFM Music Format is absolutely similar to the MUS (AdLib) format which I finished researching yesterday.

And VGFM Instrument Format is similar to SND (AdLib).

These formats were developed by Ad Lib Inc, I found the sources and documentation in the Ad Lib developer's manual downloaded from old BBS board site :) And these formats are used not only in Vinyl Goddess From Mars game.

--Binarymaster (talk) 02:05, 22 January 2015 (UTC)

Oh wow, what an interesting discovery!! Do you have any links to the official documentation? I'd really like to see what else they have to say about the format. And yes, we should definitely rename the articles here. What do you think they should be called? "Ad Lib Music Format" is probably a bit generic given they have a few, like .ROL... -- Malvineous (talk) 09:45, 22 January 2015 (UTC)

Here it is: adlip.zip - binaries and source code, probably from Ad Lib Programmer's Manual. MUS headers are described in CONVERT.C and CONVERT.H files. Fun fact that they also developed another similar format based on MIDI - MDI. It has similar musical features but stores all data in one file in MIDI format.

Well, I have no idea how to name it better, I think "Ad Lib Music Format" should be suitable for it :) --binarymaster (talk) 10:30, 22 January 2015 (UTC)

Many thanks!! I've updated the article pages with the info I could gain from the source code. There are still a few unresolved issues (like what exactly the default percussion pitch is) but it has certainly answered a number of questions. I called it "AdLib MIDI Format" because that's the name they use in one place in the source code, although it's not a very descriptive title! Thanks again for letting me know about this excellent find :-) -- Malvineous (talk) 12:48, 24 January 2015 (UTC)

That's good! I think maybe this format should be named as "AdLib MIDI Format v1.0", because there is also an improved version of it, probably v2.0 - MDI (I made a typo in previous posted link), it uses a single file in MIDI format.

And about AdLib Timbre Bank Format article - BNK format originally was created in the 1987 with the first release of the AdLib Visual Composer, so actually Ad Lib Inc released its formats in this order:

1) ROL & BNK (1987, with AdLib Visual Composer)

2) MUS & SND/TIM (1988, with player and converter)

3) MDI (1989, also with player and converter)

Information about third format was placed here, though the link is not working now.

--binarymaster (talk) 13:08, 24 January 2015 (UTC)

Oh wow that's interesting too. I think the MUS/SND/TIM format should stay as "AdLib MIDI Format" with no version number, because that file format does have version fields so it's possible to have v1.0, v2.3, etc. within that format. The MDI article you linked to actually looks like conventional MIDI with some extra events for setting OPL instruments, so I think this belongs as one of the many backwards-compatible extensions of the MID Format.

With the instrument bank formats that's really unusual if BNK came first! It seems very odd for AdLib to produce the SND/TIM format if they already had the more efficient BNK format available. I was going from the dates in the text files from the programmer's guide you kindly posted, and bkformat.txt says in Jan 1989 they are in the process of converting their software to use BNK. Can you confirm there was BNK software earlier than this? I'm just thinking that Visual Composer may have come out in 1987, but maybe it didn't get BNK support added until later? -- Malvineous (talk) 01:30, 25 January 2015 (UTC)

Well, I'm sure that first version of Visual Composer was released with AdLib music card, but I don't know exactly about BNK support. It's really strange that Ad Lib downgraded the format (BNK is better in structure), but the known formats has converters which converts BNK to X format, but not from X to BNK - this makes sense that BNK was the first format. Also Visual Composer doesn't support other bank formats, only BNK.

BNK creation date 1987 was contributed by Dean Tersigni, I'll ask him about it.

Update: I also read information from bkformat.txt, and it seems you're right :)

So actually it was:

1) ROL & INS (1987, with AdLib Visual Composer v1.0)

2) MUS & SND/TIM (1988, with player and converter)

3) BNK (1989, with AdLib Visual Composer v1.5)

4) MDI (1989, also with player and converter)

Wow, that feel when restored original order of events! :D --binarymaster (talk) 01:51, 25 January 2015 (UTC)

That seems to make the most logical sense - there's a clear improvement with each new format. I'm glad you agree! Note that when you say there's no conversion from "X to BNK" there is BANKMNG.EXE that comes with the programmer's manual which converts .ins files to .bnk. -- Malvineous (talk) 04:09, 25 January 2015 (UTC)

True. I forgot about it :) Thanks! --binarymaster (talk) 12:02, 25 January 2015 (UTC)

email

Hello! Can't send you e-mail. Got this:

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
malvineous@shikadi.net
host mail.shikadi.net [52.11.27.19]
SMTP error from remote mail server after RCPT TO:<malvineous@shikadi.net>:
550 invalid recipient

CTPAX-X Team (talk) 09:12, 11 March 2017 (UTC)

Whoops, moved my e-mail service to AWS and missed a step for that address! Please resend, should work now :-) Thanks for letting me know! -- Malvineous (talk) 11:34, 11 March 2017 (UTC)

Yes, e-mail goes through this time. Hope you receive it. CTPAX-X Team (talk) 15:41, 11 March 2017 (UTC)

Duplicate RLE page

After some edits to the RLE Compression page, I noticed that this was not the page linked from the Category:Compression algorithms page. It seems there are in fact two of them; one with an uppercase 'c', and one with a lowercase 'c'. The latter seems to be the better-written one, but since the LZW page was apparently moved from "LZW compression" to "LZW Compression", I suggest the final name of the RLE page should likewise be the uppercase-'c' one.

I have merged anything I deemed useful into the lowercase-'c' version. Could you make sure the uppercase-'c' duplicate is removed, and the other one is moved to its name?

-Nyerguds (talk) 16:48, 4 October 2017 (UTC)

Done, many thanks for the clean up! -- Malvineous (talk) 10:38, 5 October 2017 (UTC)

Account rename?

Hi Malvineous,

I am wondering if it's much of a hassle to rename accounts on this wiki. I'd kind of like to have my account use the same handle I'm using just about everywhere else (ETTiNGRiNDER). Don't sweat it too much if this is onerous to do, though.

Thanks! - Mysterioso (talk) 20:42, 29 January 2018 (UTC)

I'm not sure how it can be done. If you can find out a quick way to rename a MediaWiki account then let me know! At any rate all your comments would still be signed with your old name however. -- Malvineous (talk) 10:15, 4 February 2018 (UTC)

P.S. the e-mail address in your account is bouncing -- Malvineous (talk) 10:17, 4 February 2018 (UTC)

Ah, right, it was still set to that old e-mail. Hushmail nuked that address on me years ago, I've changed it to one that's current. As for the renaming, I don't really know anything about wiki administration, though a quick search suggests that there's this thing: https://www.mediawiki.org/wiki/Extension:Renameuser

If that's a bit much then no biggie. -- Mysterioso (talk) 14:43, 5 February 2018 (UTC)

Removal request

It seems I made a page that was unnecessary; the Westwood Palette I documented turned out to be a known common format (6-bit RGB) already covered in the standard VGA Palette page. I have changed all links to link to the older page, but I don't think I have the user rights required to delete the Westwood Palette page. Can you take care of it? -Nyerguds (talk) 14:56, 6 February 2018 (UTC)

Done! I made you an admin too so you should have delete access now. Thanks for your contributions! -- Malvineous (talk) 00:03, 7 February 2018 (UTC)

Hi-Octane

Hi!

I've figured out the MUSIC.DAT container and I was wondering if you could shed out some light on what these song formats are, now that we know how to unpack it ?

Thanks !!!

Yep, they're listed on the Hi Octane page, in the file format box. -- Malvineous (talk) 10:57, 14 March 2018 (UTC)

ReCaptcha v1 problem

It seems I can no longer add links in articles; the ReCaptcha test added when new links are detected uses ReCaptcha v1, which gives a warning that "reCAPTCHA 1 IS SHUTDOWN". It does not allow the check to pass, so the page can never be submitted.

The FAQ on the ReCaptcha docs (can't link to them, obviously) says that "Any calls to the v1 API will not work after March 31, 2018".

-Nyerguds (talk) 09:26, 28 May 2018 (UTC)

Bio Menace patches

Hey, I'll have Bio Menace 1 patches for the freeware release. Most of them are status window-related, but there are others as well. Question: Could you create several other sections in All Bio Menace patches cathegory? Because I cannot submit them anywhere right now. Thanks! --Szemigi (talk) 00:57, 25 July 2018 (UTC)

The status window patches should go into Category:Bio Menace interface patches. The others *should* fit into existing categories, but if you're not sure let me know! -- Malvineous (talk) 05:23, 29 July 2018 (UTC)

I'll need several categories from you as I checked there'll be other BM patch stuff as well. So I need the following categories: status screen, hostage, help menu, main menu, B800 screen (I even built the BIN file and I made patch for that!). And I'll even submit these for BM2 and 3. Please, create these. Oh, and I'll surely submit! --Szemigi (talk) 01:36, 1 August 2018 (UTC)

Status screen, hostage, help menu, main menu and B800 screen should all go into Category:Bio Menace interface patches as they are all modifying the user interface. I would like to keep the number of patch categories fairly small otherwise it will get a little out of control! -- Malvineous (talk) 01:48, 17 October 2018 (UTC)

Bio Menace patches on PCKF?

Hey, how can I post my Bio Menace 1-3 patches on PCKF or keenmodding.org? I very want them to submit one of the forums!--Szemigi (talk) 23:11, 4 August 2018 (UTC)

Wombat File Reader updates!

I think Wombat's title should be updated to show its full name, which now is "Wombat File Reader".

Also the supported games list has to be updated accordingly to the list on https://szevvy.com/ (the author's website, which was listed as "dead" in Wombat's info while the site is up and running again).

It's also not a dead program at all, since its latest version is very recent (11 October 2018), and it'd be a shame to have such old and deprecated information on this wiki of such a useful and powerful tool.

-Elìa1995 (talk) 07:38, 15 October 2018‎ (UTC)

It's only just been updated recently after many years of no releases. It's a bit unfair to say the wiki has old information when it's only a couple of weeks out of date, and it relies on people such as yourself to keep things up to date! Please go ahead and correct as needed so we can keep the information current. We are all doing this in our spare time so we need lots of help to keep things up to date. Also, the page you link to lists it as "Wombat File Tools" rather than "Wombat File Reader". Everyone in the community refers to it as just "Wombat" however. -- Malvineous (talk) 23:27, 16 October 2018 (UTC)

I've only just gotten my account here approved, and only just made new builds of Wombat (actually a complete rewrite) after (as Malvineous said) many many years of no releases. It warms my heart to see people notice :) Szevvy (talk) 01:44, 17 October 2018 (UTC)

I just noticed the 1.0.4 update, I'll work on updating the supported games list on the page according to the one on Szevvy's site; by the way, thanks for telling me how to "sign" these posts, I had no idea. Elia1995 (talk) 08:08, 18 October 2018 (UTC)

e-mail (again)

Hello! Sent you e-mail 2019.02.19 about Prehistorik without any answer. Yesterday another e-mail was sent and since 24+ hours passed was forced to write here. Is there something wrong again with your e-mail account or spam filters (for mail.ru)? Thanks! CTPAX-X Team (talk) 15:10, 1 March 2019 (UTC)

Content license

Is the content of this wiki governed by a specific license? I'd like to contribute information on "Nomad" file formats that I reverse engineered, which I'd previously published on my personal web site under Creative Commons SA 4.0. This wiki certainly seems to be in the spirit of the Creative Commons licensing structure so I'm guessing there would be no real conflict, but I just wanted to get some additional information on this. --Cmb (talk) 13:32, 6 May 2020 (UTC)

Personally, I've just added the license under which code was released myself. Then again I release everything under WTFPL, because I honestly don't mind people using anything I write for research purposes in any way they please, so license compatibility is a bit of a non-issue for me. -Nyerguds (talk) 16:04, 6 May 2020 (UTC)

That said, though, given the current online climate, I do agree this is an issue that should be clarified here on the wiki. -Nyerguds (talk) 16:06, 6 May 2020 (UTC)

There is a licence specified somewhere but the short answer is anything you put here is basically in the public domain - because if someone steals it we don't have the resources to chase them, so if it's that valuable to you that you can't handle someone copying it without giving you credit, then don't post it here. Having said that for "nice" users many of the pages ask for a mention if you find the content useful, but this is by no means a requirement. I suppose legally speaking you'll have to dual-licence your Creative Commons content because some CC versions have additional restrictions, like you can't copy the content without giving credit. The CC0 licence pretty much covers it. Putting a specific licence on code is fine (although no licence at all would be best so it inherits the wiki's general content licence), noting that any licence enforcement will be on you, but the content itself is always going to be CC0/public domain. -- Malvineous (talk) 05:48, 10 June 2020 (UTC)

Multiple Platforms

Should anything special be done for games that are both DOS and Win95, like Warcraft II and Command & Conquer? As it is, the template only supports one 'Platform'. I don't know enough about templates to make it accept lists, though a workaround would be to add the Win95 category to such games manually. -Nyerguds (talk) 20:29, 17 April 2022 (GMT)

I'm open to suggestions, but the purpose of the platform category is less about documenting all platforms a game runs on, and is more about identifying which games on the wiki don't work under DOS, since DOS is the wiki's main focus. So think of it more as the "minimum" platform needed - i.e. a DOS game can also run inside Win3.x and Win95, a Win3.x game can also run under Win9x but not DOS, and a Win9x game can't run under DOS or Win3.x at all. So for DOS/Windows games DOS would be the oldest platform needed to play the game, even if there is a native Windows version. -- Malvineous (talk) 00:26, 18 April 2022 (GMT)

'Minimum platform' isn't really true anymore when going to the reality of getting stuff running on modern machines, though; 16-bit windows programs don't work on modern systems at all, and DOSBox can't emulate them. They're a pretty annoying in-between class. C&C95 has issues like that; while the game itself is 32-bit, its installer is a 16-bit Windows program. -Nyerguds (talk) 10:33, 18 April 2022 (GMT)

That's true, but I wasn't thinking so much about modern systems. If you're building a VM or setting up an old PC, knowing which games will work without Windows, or will work if you only install Win 3.1, might be useful. I did think you could install Windows 3.1 in DOSBox though to run 16-bit apps. At any rate, I more had in my mind which games required OS support for NE executables in order to play, and which ones required PE support. -- Malvineous (talk) 11:48, 18 April 2022 (GMT)

Well, one of the specific cases I had in mind in this was the fact that the Win95 version of Command & Conquer uses PCX format for its title screen, while the DOS version doesn't. And while the basic modding of terrain and game objects is identical in both versions, all modding related to the game UI is different because of the different resolution. So, if more detailed information on that front would be added, it seems logical to also add the Win95 category somehow. -Nyerguds (talk) 15:17, 25 April 2022 (GMT)

In that case it sounds almost like a different game that happens to share a lot of the same content, so I'd be more inclined to list it as a separate game, e.g. on a page called "Command & Conquer (Windows version)", and just put a note saying the file formats are the same as the DOS version with the following exceptions, then list the Windows-specific file formats. -- Malvineous (talk) 15:42, 25 April 2022 (GMT)

No, it's really just PCX, nothing else is new in terms of file formats. The UI is in the same sprites format used for the in-game sprites. So like 95% still the same. But yea, I guess if I do ever get around to writing a more in-depth modding guide, I could make a different page for the Win95 version. Most of the modding difference between the two games is just that the Win95 exe file has been researched a lot deeper, to the point that (with the released source code) pretty much anything in it can be changed nowadays. -Nyerguds (talk) 16:10, 25 April 2022 (GMT)

Empty Pages

Not sure if you already noticed it, but user User:Parlortricks is creating a bunch of empty pages, I'm assuming to be filled at a later date. --TheAlmightyGuru (talk)

I hadn't noticed, thanks for the heads up! I updated ModdingWiki:Contributing to include some more points about when to create pages. -- Malvineous (talk) 04:31, 10 August 2022 (GMT)

File format category for copy-protection data.

I can't find any suitable examples with a template that suits a file containing only the copy protection data. It's not text, but there don't seem to be any generic binary format templates. Carbon14 (talk) 06:44, 17 April 2023 (GMT)

I'm not really familiar with what copy protection data would contain. What generic items would you think apply to this sort of data? I'm not so keen on adding a generic binary format category, it would be nice to have a specific one where possible. -- Malvineous (talk) 09:17, 17 April 2023 (GMT)

In this case, the file contains 750 passwords. They were printed on a piece of card in with the original game (in pale blue to thwart photocopying) and the game asks you for one of the passwords at random. I'll make a file without a specific category for now.Carbon14 (talk) 12:31, 17 April 2023 (GMT)

If they're passwords, doesn't that make it a type of strings file? You said it's not "plain text", but can it be converted/decoded to plain text? I've dealt with encrypted text formats before; see Cover Girl Strip Poker Text -Nyerguds (talk) 16:20, 17 April 2023 (GMT)

There are lots of file formats of unspecified types. See stuff like Westwood Fading Table; it's basically a file storing special colour effects. I'm not sure there's a point in making a template for every specific type like that; there's so many odd cases out there. -Nyerguds (talk) 10:17, 17 April 2023 (GMT)

Thanks. I wasn't necessarily looking for a template. But I couldn't find an example of a file that wasn't already in an incorrect template. I'll base my page on the example you suggested.Carbon14 (talk) 12:31, 17 April 2023 (GMT)

Wiki policy questions

Hi, a couple questions I've had lingering regarding how things should be done on the wiki / if they should be done at all.

What should we do about game title disambiguation? I think in cases like Warlords (SSG) vs. Warlords (Atari) where only one of the games was released as a DOS game, it's probably fine to just have a page for the one that's relevant and at most maybe have a note on its main page that it's not related to the other game, but I've noticed that "Dungeon of Death" is a title that has the honor(?) of being used for two different games on DOS. Is there a particular disambiguation format we should use when adding one of those?
Is there any specific policy on games/mods with offensive content? I know we have some strip poker type games referenced on the wiki, but one thing I'm specifically thinking of is I know there was an old hack of the DOS version of Gauntlet 2 that's been floating around since the 90s, but is a fairly puerile affair that changes text/graphics to sexual and scatological references. It also has a vulgar word in the mod title itself. It might be useful to look at how it was done since it's clearly evidence that changing the text/graphics was attainable even back then, but do we want to document things like that, or is there a "too dirty" line where we should try to keep the wiki clean of references to certain games/mods due to title or content?

Thanks.-- Mysterioso (talk) 18:35, 3 October 2023 (GMT)

Final decision on this is obviously on Malvineous, but here's my two cents on this: I don't think there's any restriction on making pages for covering games with explicit content, but it'd be nice to keep the actual explicit content on the wiki to a minimum. As an example, the most daring thing I put on the wiki was the modded showcase image for Interactive Girls Club, which, without the humorous covering, was indeed a straight up explicit image. -Nyerguds (talk) 19:23, 3 October 2023 (GMT)

Yep I agree. The purpose of the wiki is for documentation, and just like a TV documentary, sometimes you may have to cover difficult subjects, so it's usually done in a tactful way without showing anything directly offensive. So I think the same policy should apply here - no directly explicit/offensive content (e.g. images), but talking about it for the sake of documenting it is fine. I would think avoiding offensive words in article titles would be a good idea as people are more likely to come across those by accident, but if you're browsing an article about a pornographic game and you encounter some dirty words, you have nobody to blame but yourself. So listing that mod with an uncensored title is fine with me, but perhaps let's not create an article page for it unless the words are obscured with asterisks or similar just to be extra considerate to those who might be browsing through the lists of pages.

It would be good to make it clear when things are censored though. In Nyerguds' example I actually thought the 'censored' image was part of the game, I didn't realise it was an addition. So maybe it would be good to write "Censored by ModdingWiki" or something to that effect, to make it obvious that it's something we've done and not part of the original game. Likewise for an article title with asterisks in it, one of the first things on the page should be a note to say what the full uncensored article title actually is.

As for disambiguation I'm open to suggestions. Probably my first thought would be to include the year of release in brackets: "Example (1990)" like what was done for Shadow Warrior (1993 Beta) or if the year is the same, then perhaps the software developer: "Example (Microsoft)". I'm not too fussed as it's probably more important that the part in brackets makes sense to readers rather than follows any hard and fast rule. Either way a normal disambiguation page should be created for the name without any brackets, in the same style as the existing ones at Category:Disambiguation. A disambiguation page wasn't created for Shadow Warrior as the beta version isn't really a competing game, so links on the respective article pages would suffice. As always, use your best judgement, we can always change it later if we need to. -- Malvineous (talk) 06:10, 4 October 2023 (GMT)

The Interactive Girls Club image isn't censored for the sake of putting it on the wiki though. It's a showcase of the game being modded. I replaced the normal mouse cursor with that sign. I could've taken a tamer scene to start with, but I thought this would be funny. I explained what was done to it on the actual image page. -Nyerguds (talk) 07:34, 4 October 2023 (GMT)

Ohh, I understand now! Sorry it was completely lost on me, but that's probably because I've never played the game before so I didn't know what changes to look out for! -- Malvineous (talk) 10:45, 4 October 2023 (GMT)

Pages linking to itself

Hello! Merry Christmas and Happy New Year! Is it possible to check Wiki for pages that linked to itself and maybe autocorrect if possible? Like this or that? Sometimes this will result in redirect to itself (as in first example) or a red link indicating that page with this name was not created (as in second example). I've checked and fixed all pages created by MartinVole but maybe there are more. Thank you! -CTPAX-X Team (talk) 19:46, January 3, 2024‎ (GMT)

I've occasionally seen it used as shortcut to putting the page name in bold in the page intro, but yea, it's kind of odd. According to the official Wikipedia page on the subject, "Self links are usually undesirable". I immediately fixed it in the case of Shakii because, with it linking to a redirect page, it even created a real link. (PS: please sign your talk page addidions.) -Nyerguds (talk) 20:26, 3 January 2024 (GMT)

Hello and Merry Christmas and Happy New Year to you too! Wow, you fixed so much pages already! That's why I was asking Malvineous for help - maybe there is a way to do it automatically. Or at least somehow get the list of the affected pages. (Thanks! I was sure I put four "~" characters but not sure what happend and why Talk page was not included.) -CTPAX-X Team (talk) 22:35, 3 January 2024 (GMT)

Merry Christmas and happy 2024 to you too :) Yea, I'm just going down the list of all articles, heh. By the way, your talk page was not included because there was nothing there; I fixed it by coping the timestamp from the History page and adding your name manually. On that note, maybe you should make an actual user page; it's still a red link. -Nyerguds (talk) 22:51, 3 January 2024 (GMT)

I used to like using self-referential links as it meant the links would "work" again if the content was copied onto another page, but I agree once you start involving redirects it becomes a bit of an issue. So no problems from me if you remove those. There are some reports on Special:SpecialPages that list different types of page reports you can run, but I don't see one that picks up these links. I can't remember whether a self-link automatically adds the page to a category or not, because that could be another place to check, in case there is a category of all pages with self links. However I think it might have to be a case of just fixing them as we encounter them. -- Malvineous (talk) 00:35, 4 January 2024 (GMT)

Yea, I dug around a little online and found no information about detecting / listing them, so instead I just opened the list of all pages, and simply went through the whole bunch. Also fixed some pages that simply used <b> html tags instead of proper wiki formatting to make their titles bold, renamed a few pages that were missing apostrophes in their page titles, and noticed one page that still used old <code> tags. -Nyerguds (talk) 08:08, 4 January 2024 (GMT)

Wow that would've been a big job, thanks very much! -- Malvineous (talk) 09:57, 4 January 2024 (GMT)

Thank you for the answer and clarification! -CTPAX-X Team (talk) 14:51, 4 January 2024 (GMT)

Thank you for the great help with fixing the Wiki pages! Yes, user page created now, thanks! -CTPAX-X Team (talk) 14:51, 4 January 2024 (GMT)

Tilesets with "hidden" data

I was looking over the PAK Format (The Learning Company) and it made me wonder about this "Hidden" property in the Tileset Infobox template. Its function is to separate Dense tilesets from Sparse tilesets, and the main explanation given for Sparse tilesets seems to boil down to the fact that in formats that have a tiles index in their header, and some consistent way to determine the length of the data at each index, you can deliberately leave space between tiles. However, I'm not entirely clear if you consider this category to also includes block-aligned formats. See, while the PAK format indeed has an index, so it's certainly sparse, it also aligns its indices to the next multiple of 16 bytes, meaning there is potentially a lot of open space inside the format anyway. But this is not deliberately created open space caused by meddling with the index; it's just a natural result of the file format. So would a format that only has such an aligning, but doesn't have an index, be considered "sparse" too?

There's also many other ways to hide data inside a lot of formats; the simplest one being any format that uses RLE and doesn't complain about commands to repeat a given value 0 times. Does that affect the categorising? The name "Hidden" kind of gives a wrong impression in that aspect. I know it'd be a lot of edits, but "Sparse" or "Dense" would probably have been a better name for it. -Nyerguds (talk) 19:38, 8 February 2024 (GMT)

My original reason for including that information (whether data can be hidden inside the format or not) was to highlight whether there might be a way for modding tools to store additional data in the files that the game would ignore. This could be anything from a message about what program was used to generate the file, through to user comments, names/titles for items (like tiles or whole levels if the game doesn't store level names), or data that makes it easier to load the file in an editor (such as a time signature for music files that only store delays as a number of ticks, to make rendering the event list look more natural.)

So in this case the block-aligned formats would certainly qualify, as would RLE-encoded data. I figured "Hidden" was suitable because the data was hidden from the game. "Sparse" is also a little inaccurate for RLE because it's not a gap where you can put anything you want, you have to actually encode/embed the data in it. Does it matter if we call it embedded data for RLE but sparse data for tilesets? -- Malvineous (talk) 04:00, 9 February 2024 (GMT)

Well, block-aligned formats are actually probably too unreliable for really hiding in 3rd party metadata; you can never be sure there is enough space in there to reliable add extra data, since the actual data in these aligned blocks will vary. But I understand the intent better now, yes. thanks. -Nyerguds (talk) 10:47, 11 February 2024 (GMT)

FileFormatTool templates

Could you add examples and a short explanation of all the types on the BeginFileFormatTools template? I've seen pages that just use the wrong type altogether, and I'm assuming this is caused by lack of documentation. It's not clear from the page whether a tileset format containing multiple images should use type "group" or "image". I've always used "image" for that, but I can understand the confusion. -Nyerguds (talk) 10:48, 11 February 2024 (GMT)

I've added links to Template:FileFormatTool but the tileset format is a bit of a tricky one because it's kind of both. The way I've handled it in my own code is always a compromise because there's so much overlap. I've ended up deciding that if the format can hold arbitrary data (like a .zip file can) then it's an archive/group file. But many tilesets can only store image data (and often rely on headers in the tileset file to render the images within properly) so I figure those are too tightly bound together to be considered an archive.

After all, you can extract a .zip, move the files to another computer, then zip them up again and end up with a workable file. But you often can't extract all the images in a tileset, move them to another computer, then bundle them all up into the original tileset again, as data has been lost (e.g. the size of all the images which was in the tileset file's header). And you probably can't include a readme.txt in the tileset either. So therefore, it's not a true archive format. You also often can't extract an image to a file and then identify that file from its headers - tiles are often raw pixel data with no means of identification. So it's hard to open an extracted tile file in an image editor, but usually the tileset as a whole can have its format automatically detected in some way, so it's more user friendly to keep the tiles inside the tileset file, and not permit them to be extracted as raw files (only if the images are converted to another format like .png or similar). There are unfortunately still grey areas even with this definition so I'm certainly open to ideas.

Do you think the clarification belongs on Template:FileFormatTool with the rest of the info, or on Template:BeginFileFormatTools (which links there already)? Also as a side note, I have come to prefer the term "archive" over "group" so one of these days I may try to update everything to refer to "archive" instead. -- Malvineous (talk) 11:55, 11 February 2024 (GMT)

I'd add some text on the FileFormatTool page saying to look at the BeginFileFormatTools page for explanation on the types themselves, and then write the real stuff there. After all, the "Begin" template is the one that contains the type option, so that's where I would expect the explanation, as user.

And, yea, changing "group" to "archive" might be a good move; it makes it more distinct, and it's already used like that anyway. There will of course always be some edge cases, but there's not much we can do about that. It's generally archives containing mostly but not exclusively images, and which don't have names for their entries. All Dogs Archive Format comes to mind; despite the name, when writing the page I treated it as a tileset with one type of special non-image entries. -Nyerguds (talk) 18:17, 15 February 2024 (GMT)

Done, hopefully that provides some better documentation. Feel free to adjust as needed. -- Malvineous (talk) 06:12, 18 February 2024 (GMT)

Absolutely beautiful. Great work on that. I might take some time referencing the relevant Infoboxes for each one, in a "This type is generally used on pages using the xxxx and yyyy infoboxes" way, since these correspond pretty directly -Nyerguds (talk) 08:57, 1 March 2024 (GMT)