Jump to content

Edit Style
Primary: Sky Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Secondary: Sky Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Pattern: Blank Waves Squares Notes Sharp Wood Rockface Leather Honey Vertical Triangles
Welcome to Fuwanovel Forums
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account

Best Wishes to Krill

Please take a moment and post a message of support for Krill.

Photo

Data extraction thread

Data Sprites BGM Music Backgrounds Extract Extraction

256 replies to this topic

#1
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts
Long time idea. If for whatever reason you need some data from a game, would it be music, sprites, CG's and if like me you SUCK with computers, well worry no more because I have the solution for you.

How is this thread going to work ?

- All the info is going to be centered in the two first posts, first one will be File types, second one will be Versatile tools.

- I'm going to explain the process to extract for each file type. If you don't know what type a VN uses, look into your game file dummy!

- However, some applications were specificly made to extract files from some Visual Novels in particular. If that is the case it will be specified.

- Contributions are highly appriciated. Either PM me with the extraction process thoughfully explained or post in this thread with the same kind of format that I will use.

- If ever you extract files from a game which wasn't verified or not listed with one of these methods, please report it and tell us what was is in each file. Thank you.

I am not a coder, none of these applications are mine. The sole purpose of the thread it to regroup the scatered information you may hardly find online. PM me if there is a link problem.
 

_____________________________________________________________________

File Types

.xp3 Files
Spoiler


.rpa Files
Spoiler


.int Files
Spoiler


.arc Files
Spoiler


nscript.dat and arc*.nsa Files
Spoiler



.pak Files
Spoiler


.bmp Files
Spoiler


.npa Files
Spoiler


.bin Files
Spoiler


.nwa Files
Spoiler


.G00 Files
Spoiler


.ks Files
Spoiler


.rpd Files
Spoiler


.dat Files
Spoiler


.pd Files
Spoiler


.noa Files
Spoiler


.afd & .alf Files
Spoiler


.gxp Files
Spoiler


.rio Files
Spoiler



.ald & .alk & .alm Files
Spoiler



.ain Files
Spoiler



.rct & .rc8 Files
Spoiler



.nds Files
Spoiler


.sar Files
Spoiler

_____________________________________________________________________


sZN1zC9.jpg


#2
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Versatile tools

The tools mentionned here can be used to extract a very wide variety of archives.
If you do not find a type in the list above, give these a try.
If they succesfully work for a certain format and game, please PM me with the details, thanks. (It may help others in the future)
(I'll update later with a precise list of their functionnality)

Crass
Spoiler


AnimED
Spoiler


Susie32
Spoiler


arc_conv
Spoiler


Asmodean's tools
Spoiler


ExtractData
Spoiler

_____________________________________________________________________

Visual Novels

Fuwanovel's
Spoiler


Others
Spoiler


_____________________________________________________________________

Useful Websites

Asmodean's Reverse Enginneering page - tools

WinKiller Studios (AnimEd)

TLwiki - tools?

Digitalpag (Susie)

I-forge.net - tools

Proger's blog - hack

My CG rip Diary - tools

My CG Ripping tools

_____________________________________________________________________

Endless Work in progress

Thank you very very much toZakamutt,Down, Blizzard884,The Major, Steve & Luch, dsp2003, Binaryfailfor their major help in the project (pun intended).

PS : will add more pictures eventually, just too lazy right now

PPS :If you know where to download Susie plug-ins, please tell me. Most of the links are old or dead. If you still have some of them, uplaoding them would help greatly. Thank you

Edited by maefdomn, 03 April 2015 - 11:07 AM.

sZN1zC9.jpg


#3
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Hex 


sZN1zC9.jpg


#4
Zakamutt

Zakamutt
  • 「BEST GIRL」だね。うん、「BEST GIRL」。

  • PipPipPipPipPipPipPipPipPipPip
  • 2321 posts

Missing kikiriki for .xp3. Admittedly slightly more unfriendly because it's a command-line tool, but shouldn't be too hard. This is where I got most of my info for it - download here. Not sure whether it does anything special to break encryption or whatever though, anyway... place kikiriki and the dll it comes with in the vn's root directory, open a command prompt there - shift + right-click inside the folder in windows explorer brings up a handy option for this in win7. Download page shows you how to use it, but here's an example of practical usage:
 
kikiriki.exe -i data.xp3 -o d:\VNs\examplefolder\
 
This successfully extracts the data for World End Economica (which I just used to test) into D:\VNs\examplefolder\. Remember, I had to open my command prompt in the directory WEE was in for this to work (advanced usage omitted).
 
Tools galore: http://tlwiki.org/in...php?title=Tools

 

Yeah there are several tools for xp3. There's suzie too.
But Crage handles so many Visual novel that's it often worth trying it first.
 
I've been running test for hours and now Crage decided to crash my computer whenever I extract, making this extra painful.
 
I'll add your method, if you could send a couple pictures to illustrate the thing that would help me.


  

Wow meaf, you have taken on a bit project  :) It will be a lot of work, its not really as simple as one file extension uses the same procedure, different versions of kirikiri use different version of xp3 archive, you need different tools for each of them.
 
You should use this http://haibara4cgrip...s-v2-7-english/
Its the latest version of the guide, he updates it about once a year
 
It still doesn't have all the available tools (lot of them are made specifically for one game, but can work with games on the same version of engine) but it has a lot so you can start there.
 
 
But if you actually finish this project then props to you  :) But it will be hard, very very hard.

 

Oh one more thing, not sure if you will be also including extraction of scripts - extraction of CG/music is pretty straightforward most of the times, but extraction of scripts has another thing on top of it such as obfuscated scrtipts through bitshifting and whatnot.

For now i'll stick to the "easy" stuff. But some script file types are pretty redundant (such as ks files) and it way be worth mentionning them.
It will be done ... eventually ... ;p

 

Images for kikiriki: http://imgur.com/a/K5dzv
 
Notably WEE has no encryption as all from what I can tell, insertion has worked perfectly for me so far. Don't try to create the archive in the same folder kikiriki is though, it causes cryptic errors (even if you don't name it data.xp3 which has obvious name collision problems)
 
I'm not responsible for naming my D drive warcraft. Blame my dad.


Edited by Tiagofvarela, 10 April 2015 - 01:47 PM.
Hogging this post to try and save some older ones.

pDAhAwK.png
The Fuwazette's editor-in-chief! Personal blog: Disearnestly Disearnest. Ask me anything here.
Click the like button in the bottom right if you enjoyed my post. It's not a social media share link.


#5
The Major

The Major
  • President of Pizza

  • 513 posts

I think this is something MangaGamer uses a lot. My example here is with Deardrops.

 

I used the ExtractData tool, which can be found on the page you linked too. I specifically used the "Improved English version", not sure if it makes a difference. Probably not.

 

.arc Files

Spoiler



#6
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Awesome, Thanks a lot. Added !

In Deardrops.

Do you know which files contain what ? Or is it all mixed up ?


sZN1zC9.jpg


#7
Down

Down
  • Hater of Fun

  • 2732 posts

For NScripter engine:

 

nscript.dat and arc*.nsa Files

Spoiler

 

If someone wants to test other NScripter based engines...


I write VN reviews for Fuwa Reviews. Otherwise I'm too lazy to write stuff.


#8
Vokoca

Vokoca
  • ゆっくり流の達人

  • PipPipPipPipPipPipPipPip
  • 1044 posts

Hm, tried the Crage method for AkaGoei Trinity, and while it did extract the BGMs in .ogg format, I can't play any of them..?


j99egp.jpg


#9
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Hm, tried the Crage method for AkaGoei Trinity, and while it did extract the BGMs in .ogg format, I can't play any of them..?

 

Don't ogg files open with VLC media player ?

 

Awesome Down, Thanks a lot, I'll add that now.

Could you add a picture of the result.txt file ? It's kinda useless but an image allows the text to be a little easier to read :P.

 

I did the Grisaia .int exception, now to polish it.


sZN1zC9.jpg


#10
Zakamutt

Zakamutt
  • 「BEST GIRL」だね。うん、「BEST GIRL」。

  • PipPipPipPipPipPipPipPipPipPip
  • 2321 posts

Pretty much everything can be played by foobar2000, moderately recommended (though the UI and certain other thigns leave much to be desired...)

VLC takes way too long to launch on audio since it's a video player in my experience


pDAhAwK.png
The Fuwazette's editor-in-chief! Personal blog: Disearnestly Disearnest. Ask me anything here.
Click the like button in the bottom right if you enjoyed my post. It's not a social media share link.


#11
The Major

The Major
  • President of Pizza

  • 513 posts

Awesome, Thanks a lot. Added !
In Deardrops.
Do you know which files contain what ? Or is it all mixed up ?

 
data01000.arc - files? I don't know what they are. I'll attach an image

Spoiler

data02000.arc - sprites and other in-game images, like from lewdy scenes and whatnot
 
data02999.arc - some .bmv files. I don't know what these are.
 
data03000.arc - sounds effects
 
data.04000.arc - voice tracks
 
data.05000.arc - music and some other sound effects

sysgrp.arc - Images used for the menus and interface

sysprg.arc - ._bp files? I don't know what these are.

syssnd.arc - two audio files, the mouse click sounds for menus

system.arc - ipl._bp, launcher._bp, VeraMoBd._ft. I have no idea what these are.



#12
Vokoca

Vokoca
  • ゆっくり流の達人

  • PipPipPipPipPipPipPipPip
  • 1044 posts

Don't ogg files open with VLC media player ?

They should, but it doesn't work for some reason. Weird. Well, I can just go look for the OST somewhere either way.


j99egp.jpg


#13
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

They should, but it doesn't work for some reason. Weird. Well, I can just go look for the OST somewhere either way.

 

Strange, it works perfectly for me.

Oh and if you Crage'd AkaGoei Trinity could you quickly try the other xp3 files and tell me what's in them ? 

Please 

Spoiler


sZN1zC9.jpg


#14
Down

Down
  • Hater of Fun

  • 2732 posts

Could you add a picture of the result.txt file ? It's kinda useless but an image allows the text to be a little easier to read :P.

Here's an explanation pic:

Spoiler


I write VN reviews for Fuwa Reviews. Otherwise I'm too lazy to write stuff.


#15
Clephas

Clephas
  • Infinite Stomach

  • PipPipPipPipPipPipPipPipPipPip
  • 2196 posts
  • LocationERROR, ERROR, FAILURE TO OBTAIN CONCLUSIVE DATA
  • VNDB

Try Asmodean's reverse-engineering download page.  He has ones for hundreds of VNs on that page.


肉、肉、肉! この世界は肉でできている! われは捕食者の神、弱肉強食の王、すべての世界を食べるものなり! 創造、無限胃袋!

Spoiler

#16
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Try Asmodean's reverse-engineering download page.  He has ones for hundreds of VNs on that page.

 

Yeah I know of his page, but many are incredibly specific.

This is where I got the Grisaia Image extractor for example.

 

But yeah, I'll look into this for sure. Thanks.

 

And thanks Down, I didn't even realize it was the original Umineko. I'll add it.


sZN1zC9.jpg


#17
Steve

Steve
  • Loli Protector

  • PipPipPipPipPipPipPipPipPipPip
  • 2155 posts
  • LocationColorful World
  • VNDB

Yeah I know of his page, but many are incredibly specific.

 

Well they need to be very specific since they differ from game to game. Crass just has all that mixed together.

However crass and similar universal tools are dead projects for many years now so it doesn't work with lot of new releases.

 

Westside and asmodean are probably the closest you get to something like crass since its all centered on one website and generally the commands are similar. Those 2 update constantly for even the newest releases.


FHtrfPd.png
R.I.P. Fuwa Freedom Of Speech, 2012 - 2015

#18
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

I'm having issue with .pak files.

Even though it seems fairly common the only free tools i've found are either outdated or simply do not work.

These do not work, I tested them on Symphonic rain and Dra+KoI

- Winrar

- Alzip

- pakexplorer

- paxexlpr


sZN1zC9.jpg


#19
Zakamutt

Zakamutt
  • 「BEST GIRL」だね。うん、「BEST GIRL」。

  • PipPipPipPipPipPipPipPipPipPip
  • 2321 posts

^ http://tlwiki.org/in...lic_Serenade.29 for SR


pDAhAwK.png
The Fuwazette's editor-in-chief! Personal blog: Disearnestly Disearnest. Ask me anything here.
Click the like button in the bottom right if you enjoyed my post. It's not a social media share link.


#20
The Major

The Major
  • President of Pizza

  • 513 posts

Just figured out .npa files. I'm pretty sure this is what Nitro+ uses all of the time. my example here is with Saya no Uta, and I'll try it later with my copy of Steins;Gate.

 

The tool I used is NIPA (https://github.com/W...n/nipa/releases)

 

.Npa Files

 

Download nipa.exe

 

Place nipa.exe in the same folder as the .npa files you want to extract data from

 

Open command prompt, then change directory to where you have the .npa files and nipa.exe

 

example

cd "c:\Users\username\Desktop\Saya no Uta"

Then in command prompt, type nipa -x filename.npa

 

example

nipa -x sound.npa

This will extract the contents of sound.npa into a new folder named "sound"



#21
The Major

The Major
  • President of Pizza

  • 513 posts

As for the contents of Saya no Uta:

 

cg.npa - all images

 

nss.npa - a bunch of .nss files. I don't know what these are.

 

sound.npa - all sound effects and music

 

voice.npa - all voice tracks

 

system.npa - system.ini



#22
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

Thanks a bunch guys !

 

Zak, did you find a working version of Susie ?

I downloaded it here and extracted the thing but the toolbar isn't launching.

It's a really old software, I launched it using XP compatibility but that didn't change a thing.


sZN1zC9.jpg


#23
Vokoca

Vokoca
  • ゆっくり流の達人

  • PipPipPipPipPipPipPipPip
  • 1044 posts

Just figured out .npa files. I'm pretty sure this is what Nitro+ uses all of the time. my example here is with Saya no Uta, and I'll try it later with my copy of Steins;Gate.

 

The tool I used is NIPA (https://github.com/W...n/nipa/releases)

 

.Npa Files

 

Download nipa.exe

 

Place nipa.exe in the same folder as the .npa files you want to extract data from

 

Open command prompt, then change directory to where you have the .npa files and nipa.exe

 

example

cd "c:\Users\username\Desktop\Saya no Uta"

Then in command prompt, type nipa -x filename.npa

 

example

nipa -x sound.npa

This will extract the contents of sound.npa into a new folder named "sound"

I hate to be the guy who couldn't get it to work again, but I think I can confirm that this doesn't work with Muramasa. Can't handle the encryption, it seems.


j99egp.jpg


#24
maefdomn

maefdomn
  • Admin

  • PipPipPipPipPipPipPipPipPip
  • 1850 posts

I hate to be the guy who couldn't get it to work again, but I think I can confirm that this doesn't work with Muramasa. Can't handle the encryption, it seems.

 

Tested Steins;gate, I get a Invalid NPA archive error.

 

Edit : Same for Choas;head

 

i can confirm that it works for Saya no Uta


Edited by maefdomn, 26 May 2014 - 07:00 PM.

sZN1zC9.jpg


#25
The Major

The Major
  • President of Pizza

  • 513 posts

Tested Steins;gate, I get a Invalid NPA archive error.

 

Edit : Same for Choas;head

 

I hate to be the guy who couldn't get it to work again, but I think I can confirm that this doesn't work with Muramasa. Can't handle the encryption, it seems.

 

I probably should have mentioned to take a look at the Usage (https://github.com/W...nsen/nipa#usage) and Supported Games (https://github.com/W...supported-games) section on the nipa project page. For Chaos;Head you'll want to do nipa -xg filename.npa ChaosHead, and nipa -xg filename.npa Muramasa for Muramasa. Even though Saya no Uta isn't on the Supported Games list, it still works for some reason.

 

I can confirm that Steins;Gate doesn't work. I was looking around, and according to this page (https://github.com/FGRE/libnpa#libnpa), "There are 4 known .npa file formats, three of which can be read by nipa tool. This library (currently) implements fourth which is used by Steins;Gate". So apparently Steins;Gate uses a different kind of .npa which NIPA can't read. So Steins;Gate wanted to be different, ruining the fun for the rest of us.







Also tagged with one or more of these keywords: Data, Sprites, BGM, Music, Backgrounds, Extract, Extraction

1 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users


    Alexa (1)