PDA

View Full Version : Can unpacking the bigfile.00X files run the game better?



eastcoasthandle
1st Sep 2011, 21:49
Hello,

Can a developer comment if unpacking the bigfiles will allow the game to run smoother, faster if you have an high end PC? If so, could a developer be so kind as to comment on how to do it? Or, if not explain any complications.

Saloei
1st Sep 2011, 22:28
good question! im experiencing the jitters when scenes load and find it to be distracting. if this fixes the jitters, then its well worth it. please respond.

eastcoasthandle
2nd Sep 2011, 02:08
I hope a developer will be so kind as to address this. Theoretically, unpacking the bigfiles so that you have several sub-directories and files should smooth out the game because it's not decompressing and temp. storing data elsewhere (just my theory not necessarily what's going on).

LordWeasel
2nd Sep 2011, 10:21
I remember unpacking the game files was a fix for some kind of issue with Quake 1 back in the day. Hey, maybe this fix is an oldie but a goodie :P. Would love to be able to try it.

Tricksterthegreat
2nd Sep 2011, 10:47
Unfortunately it just seems to give you an error with no text

Maybe someone else can give it a shot

Moork
2nd Sep 2011, 14:00
Hurrah for the modders! If someone CAN make this game moddable by unpacking the files, maybe we can get some fixes for the games annoyances. I always find modders make better patches than the devs themselves.

eastcoasthandle
2nd Sep 2011, 14:15
Unfortunately it just seems to give you an error with no text

Maybe someone else can give it a shot

I wouldn't think that unpacking the bigfiles alone would actually work as you would need to modify the game's .exe to recognize that the game is now decompressed.

weevil
2nd Sep 2011, 23:07
how would you go about unpacking the .00x files?

motsm
3rd Sep 2011, 01:00
how would you go about unpacking the .00x files?http://forum.xentax.com/viewtopic.php?f=10&t=7208

eastcoasthandle
4th Sep 2011, 00:56
:hmm:
Would a developer or whoever is involved be so kind to address this thread regarding the bigfiles?

Also, here is an mod for the game I found in this forum. It doesn't answer the OP question but thought it was something worth noting none the less.
http://forums.eidosgames.com/showthread.php?t=121245

eastcoasthandle
5th Sep 2011, 18:11
I'm still looking for some insight on the decompression of the bigfiles.

Brockxz
5th Sep 2011, 18:57
I remember unpacking the game files was a fix for some kind of issue with Quake 1 back in the day. Hey, maybe this fix is an oldie but a goodie :P. Would love to be able to try it.

For Doom 3 unpacking .pak files reduced loading times and improved overall performance.

NixxesSoftware
5th Sep 2011, 20:21
Unpacking the files is not something that would improve performance in any way.

Splitting the files into many smaller files would only increase OS overhead as the game is accessing the files. Performing operations on many thousands of files is simply slower as only operating on one larger file.

Decompressing the data in theory could improve performance, but in practice would just mean that it takes longer to load the data from disk. Decompressing data on the CPU is entirely asynchronous and is not a significant performance hit for todays CPUs.

Tricksterthegreat
5th Sep 2011, 20:25
What a shame.

eastcoasthandle
6th Sep 2011, 01:43
Unpacking the files is not something that would improve performance in any way.

Splitting the files into many smaller files would only increase OS overhead as the game is accessing the files. Performing operations on many thousands of files is simply slower as only operating on one larger file.

Decompressing the data in theory could improve performance, but in practice would just mean that it takes longer to load the data from disk. Decompressing data on the CPU is entirely asynchronous and is not a significant performance hit for todays CPUs.

Thanks for answering the question :thumb:

Nixxes,
Do you think that TSC vs HPET would make any difference in performance for this game when using win7? Using "bcdedit /set useplatformclock true" does seem to help with some of the stuttering. Would a timing issues help fix the stuttering?

eastcoasthandle
6th Sep 2011, 03:15
Hmm, after playing some more the HPET fix does make an improvement to the game. The load times are a tad quicker and it's not stuttering as much as it did before. Nixxes, can a timing issue be the cause of the stuttering?

eastcoasthandle
6th Sep 2011, 13:28
Nixxes,
I'm not sure if you caught my reply to your post or not but I wanted to tell you that enabling HPET (From TSC) on Win7 does seem to help. Could you look into it?

Remnant
6th Sep 2011, 14:15
bcdedit /set useplatformclock true

Tried it, load stutter still present and not better at all IMO.

But IDK if that changed anything on my system, ICH9 chipset is said to support this but I don't have an option for HPET in my bios. :\

NixxesSoftware
6th Sep 2011, 14:28
HTEP was designed to provide better timing results on operations systems and hardware that support it, compared to TSC. Deus Ex does use the timing functions in Windows that are affected by this, so from that perspective it is not impossible for it to help.

That said, the main problem that HTEP addresses (as far as we know) is fluctuating timers between cores, and Deus Ex should not be affected by that, all relevant timing happens on a single core. I certainly wouldn't expect it to impact load times.

But again, HTEP is a better and more accurate timing mechanism, and it probably should be enabled for all operations systems that support it.

Remnant
6th Sep 2011, 14:36
I've asked before and didn't get a response, so maybe you'll catch this one:

Is there any way we can dump an API call trace exhibiting the issue?

AMD's GPU PerfStudio has nice low-level tracing options, but I can't get it to launch the game properly (I think steam takes over and relaunches it). Even if the problem isn't GPU related, the API calls leading up to the stalls might give you clues where to look.

eastcoasthandle
6th Sep 2011, 14:58
bcdedit /set useplatformclock true

Tried it, load stutter still present and not better at all IMO.

But IDK if that changed anything on my system, ICH9 chipset is said to support this but I don't have an option for HPET in my bios. :\
I have no idea why you did it and then expected to work without rebooting. However, I never gave specific details for users to try it, remove it, etc. Furthermore, I would assume that your motherboard would need to support HPET (Nixxes can chime in with any other details). It was only directed for the developer :p.



HTEP was designed to provide better timing results on operations systems and hardware that support it, compared to TSC. Deus Ex does use the timing functions in Windows that are affected by this, so from that perspective it is not impossible for it to help.

That said, the main problem that HTEP addresses (as far as we know) is fluctuating timers between cores, and Deus Ex should not be affected by that, all relevant timing happens on a single core. I certainly wouldn't expect it to impact load times.

But again, HTEP is a better and more accurate timing mechanism, and it probably should be enabled for all operations systems that support it.
After doing this the game does perform better (not saying that stuttering is completely removed but better). I can't explain why that is other then report what I'm seeing to you. But I do thank you for the information.

Remnant
6th Sep 2011, 15:05
I have no idea why you did it and then expected to work without rebooting. However, I never gave specific details for users to try it, remove it, etc. It was only directed for the developer :p.

The BCDEDIT program modifies boot time settings, so of course I rebooted.

And there cant be any harm in testing these sorts of things just for the fun of it.

eastcoasthandle
6th Sep 2011, 15:07
The BCDEDIT program modifies boot time settings, so of course I rebooted.

And there cant be any harm in testing these sorts of things just for the fun of it.

You obviously don't know what your doing. Perhaps if you asked about it then I would have given more details. However, you are still missing a crucial part of this. And because you've not mentioned what it is I'll leave it at that for now while you play the "know it all" kind of replies.

Remnant
6th Sep 2011, 15:17
Tone it down, were trying to troubleshoot an issue here. If I came across as a know it all, it was only in vain of the fact that your post was fairly absent of details on what you did. So please, all knowledgeable one, enlighten us to your supreme ways. In all fairness, willingly withholding information and telling me I have no idea what I'm doing just makes you look like the "know it all." You should want people to test your theories, just don't get all butthurt when someone calls out your placebo.

EDIT: I ran some tests with it enabled and disabled and it looks like it does work on my system, I'm seeing differences in that timer performance app from nvidia. Yeah, I found your post on sevenforums. If theres anything else I'm missing, I'm all ears.

NixxesSoftware
6th Sep 2011, 15:54
Thanks for the offer Remnant, but we do not expect to need any GPU logs from your machine.

eastcoasthandle
6th Sep 2011, 16:11
Thanks for the offer Remnant, but we do not expect to need any GPU logs from your machine.
I'm curious to know how you have your PC's setup. Are they with TSC or do they use HPET


HTEP was designed to provide better timing results on operations systems and hardware that support it, compared to TSC. Deus Ex does use the timing functions in Windows that are affected by this, so from that perspective it is not impossible for it to help.

That said, the main problem that HTEP addresses (as far as we know) is fluctuating timers between cores, and Deus Ex should not be affected by that, all relevant timing happens on a single core. I certainly wouldn't expect it to impact load times.

But again, HTEP is a better and more accurate timing mechanism, and it probably should be enabled for all operations systems that support it.
You still here, lol? I have no idea what you are babbling about but what you replied with isn't what I'm referring to. But thanks for proving to me that you don't know what your doing and more then likely been goggling for an answer. Must be hard for you to swallow your pride in all this. But needless to say based on your conduct towards me you won't get any of my help. Enjoy your day? :wave:

Remnant
6th Sep 2011, 16:16
^^lol. Next time just keep your fail to yourself, eh?


Thanks for the offer Remnant, but we do not expect to need any GPU logs from your machine.

Well I suppose thats good news :)

Does this mean you already have a strong lead on how to fix it? I'm not pushing for an ETA, these things take time to cook of course, but a "yeah we found it" would be nice to hear right about now.

eastcoasthandle
6th Sep 2011, 19:32
After trying the alley way behind the gas station I'm not seeing the same kind of stuttering I once noticed before. So there is some improvement there. I still wonder if the developers PC's were setup for TSC or HPET?

Remnant
7th Sep 2011, 02:44
I would hope they tested on both setups considering that they are 'in the wild.'

eastcoasthandle
7th Sep 2011, 15:23
Here is a quick video I did showing the alley way with HPET enabled.
QMh4FsrhPAY
No noticeable stuttering even though I'm still using fraps. It's not 100% but it is noticeably better then before.

LordWeasel
7th Sep 2011, 16:18
Just tried enabling HPET, and does seem to reduce the stuttering. It's not gone by any means though.

eastcoasthandle
7th Sep 2011, 16:50
Here is a walk/run through some of Detroit. Stuttering reduced.
dKL4W8WFOSE

WaDn8XWgfnc

Remnant
7th Sep 2011, 17:54
Maybe it was worse for you before, but thats about how it runs for me on either timing. Its just so much more visible when at full res and framerate. For me, theres a couple corners in particular where the load stutter is exceptionally bad regardless of settings.

Seems like this might offer some machines a similar improvement as lowering graphics quality has been noted to do.

As said, it isn't a fix by any means. I think it just shows that the engine isn't timing is interrupts or DPCs (or maybe even context switches) properly, and a finer grained timer just helps it a bit.

eastcoasthandle
7th Sep 2011, 17:58
Just tried enabling HPET, and does seem to reduce the stuttering. It's not gone by any means though.

At this point the game is completely playable. The only time it stutters is when you see that animated disc which pops up whenever it's saving/loading or whatever it does. :thumb:



Maybe it was worse for you before, but thats about how it runs for me on either timing. Its just so much more visible when at full res and framerate. For me, theres a couple corners in particular where the load stutter is exceptionally bad regardless of settings.

Seems like this might offer some machines a similar improvement as lowering graphics quality has been noted to do.

As said, it isn't a fix by any means. I think it just shows that the engine isn't timing is interrupts or DPCs (or maybe even context switches) properly, and a finer grained timer just helps it a bit.
What you are seeing is Fraps capped at 30 FPS. So what you are saying is that the game to you feels like 30 FPS. Which is not the case for me. Any stuttering you claim to see is the result of Fraps not what I see in game when playing. In any case, I thought I would provide videos since you were so certain that there was no difference for me. "...Next time just keep your fail to yourself, eh?" :poke:

Remnant
7th Sep 2011, 18:12
What you are seeing is Fraps capped at 30 FPS. So what you are saying is that the game to you feels like 30 FPS. Which is not the case for me. Any stuttering you claim to see is the result of Fraps not what I see in game when playing.

Yeah, a machine capable of running it at 50-60 sure shouldnt feel like its in the 30s. And it indeed does sometimes. But as you admit, fraps+youtube is introducing enough unknowns that they don't make the best benchmark for this sort of thing. And how it "feels" to us is an entirely subjective benchmark. I'm still seeing the ~.5 second stalls when turning some corners, doesn't matter if it was autosaving or not.

Even if this tweak does improve things for some machines it doesn't excuse the fact that every other game around runs fine without it.

I got a weird hunch, maybe someone with enough RAM could load the game files into ramdisk with this tweak and see what happens. Not that its any sort of a practical solution, but if this timing tweak moves the stalls closer to the HDD loading then a ramdisk should minimize that. Might need to move the pagefile to ram too though, I think the game uses a decent amount of that regardless of how much memory there is available.

jonB89
7th Sep 2011, 21:47
Meh... didn't really see a difference when enabling HPET to 64-bit. I was REALLY hoping it would be a dramatic improvement. :(

Remnant
8th Sep 2011, 03:50
Meh... didn't really see a difference when enabling HPET to 64-bit. I was REALLY hoping it would be a dramatic improvement. :(

Its unfortunate indeed. Best to not get your hopes up for these types of quick fixes. Sometimes one comes along and does the trick but it isn't often. This guy (http://forums.steampowered.com/forums/showpost.php?p=24890114&postcount=581) sums it up pretty well. Be vigilant, there will probably be more of these "fixes."

eastcoasthandle
8th Sep 2011, 13:23
HPET offered in this thread was not suggested as a fix for everyone. As a matter of fact there is no detail explanation on how to implement HPET, how to disable it, etc for a reason (read post 21). As I stated earlier in this thread, which is labeled "Can unpacking the bigfile.00X files run the game better?" this suggestion was offered to Eidos directly to see if their is something that can be added to the patch (read my earlier posts).

The videos were provided to offer proof that it is working on my system and to shutup Remnant bickering that it wasn't a fix for me. In which he changed his tune afterwards, lol. But only to show that he didn't comprehend that the videos provided only showed a frame rate of 30 FPS and mistaken that as in game stutter. When in fact the game is not played that way.

Remnant
8th Sep 2011, 14:23
In your first video your run through you were stopping at corners and turning around, retracing your steps. Id rather you keep running and turn the corner, thats the proven way to trigger this issue.

You're second video clearly displays it at 55-60 seconds when you round the corner at the dumpsters. I watched it at least 5 times through that area. Its just a tad harder to see given the low fraps/youtube framerate.

I've heard that setting the present interval to 2 (half of v-freq) via registry setting hides the stutter a bit but its still there, so fraps capturing at 30 could cover it up in a similar way.

eastcoasthandle
8th Sep 2011, 14:41
In your first video your run through you were stopping at corners and turning around, retracing your steps. Id rather you keep running and turn the corner, thats the proven way to trigger this issue.

You're second video clearly displays it at 55-60 seconds when you round the corner at the dumpsters. I watched it at least 5 times through that area. Its just a tad harder to see given the low fraps/youtube framerate.

I've heard that setting the present interval to 2 (half of v-freq) via registry setting hides the stutter a bit but its still there, so fraps capturing at 30 could cover it up in a similar way.

At the 55-60 mark I actually did stop there. However, fraps seems to have recorded it a tad different then in game. ;)

eastcoasthandle
8th Sep 2011, 15:11
You're either oblivious or a dedicated troll. It doesn't work. Get over it.
I'm only going to ask you once, please stop bickering in my thread. I've answered your question.

Remnant
8th Sep 2011, 15:16
You can make up an excuse for the second vid, but the way you were walking around in the first (of post 32) looked like you were intentionally avoiding the trigger spots. You didn't answer anything.

I'll ask you again, make another vid, recording from game launch as to rule out any 'manual caching' you may be doing. Run around those corners in the subway station like you should have, and how a player would normally pass through that area.

eastcoasthandle
8th Sep 2011, 15:47
You can make up an excuse for the second vid, but the way you were walking around in the first (of post 32) looked like you were intentionally avoiding the trigger spots. You didn't answer anything.

I'll ask you again, make another vid, recording from game launch as to rule out any 'manual caching' you may be doing. Run around those corners in the subway station like you should have, and how a player would normally pass through that area.

Your bickering is doing nothing more then inciting an argument. Your bickering is clearly debunked because the alley way videos run through is provided twice already in this thread. The very 1st video doesn't exhibit the same behavior as the one you mentioned. So:
-the 1st video debunks your theory
-I've answered your question how the pause was created in the later video

I find no reason to argue with you about it as you're only coming up with unfounded excuses. Just show you the evidence I was posting about earlier. And, show it to Eidos to see if it could help them with a patch. :)

mazuma
10th Sep 2011, 19:04
Bump*

Has anyone been able to succesfully unpack the game yet ?

eastcoasthandle
11th Sep 2011, 22:00
Bump*

Has anyone been able to succesfully unpack the game yet ?
I have not seen any more information about it.

----------------------------------------------
More information about HPET (Discussing Win7)
----------------------------------------------

Since I'm reading a post (http://forums.eidosgames.com/showpost.php?p=1681455&postcount=26) regarding HPET being enabled here is a way to remove it if you need to:
bcdedit /deletevalue useplatformclock (then reboot)

mazuma
11th Sep 2011, 22:11
I have it enabled atm, and tested with both on and off dint make a diffrence for me unfortunatly.

Might work for others though cant hurt to try ..