Out of memory error processing big files

Moderator: jsachs

Post Reply
pawel
Posts: 2
Joined: April 28th, 2009, 4:45 am

Out of memory error processing big files

Post by pawel »

Hello,
Just to check whether scanning at 5400 dpi can give a real quality boost I started processing huge (226 MB) 16 bit TIFF files in PWP. I noticed several problems:
  • PWP cannot handle more than about 8-10 open files concurrently.

    Sometimes when using workflow (with 30-40 226 MB files added) PWP starts to return "cannot read TIFF file" error when trying to set widget or open individual file (and once the error occurs PWP refuses to open ANY file) - after PWP restart the problem is gone. I haven't noticed this error when using workflows with literally hundreds of 56 MB TIFF files.
I'm using PWP 5.0.1.6 on Windows XP Professional with 2GB of RAM. Is any update planned to PWP relating to its performance when using huge TIFF files?

Pawel

ksinkel
Posts: 593
Joined: April 2nd, 2009, 11:58 am
Contact:

Re: Out of memory error processing big files

Post by ksinkel »

The problem is that memory in 32 bit Windows operating systems is restricted to a total of 2GB. We will be introducing a 64 bit version of PWP in the new year. Using that and a 64 bit version of Windows will give you much more address space for large images.

Kiril
Kiril Sinkel
Digital Light & Color

pawel
Posts: 2
Joined: April 28th, 2009, 4:45 am

Re: Out of memory error processing big files

Post by pawel »

Thank you for such a quick reply - looks like time for another computer and OS upgrade is coming. :-(

However, lack of address space is one thing and relative PWP instability when processing large files is another. I just have had to restart PWP after several 226 MB images processed because it allocated 2GB of RAM and refused to open any file ("Out of memory" error) even when there were no files - apart from workflow window - opened. Do you plan to investigate such issues with current 32 bit version of PWP? How can I help?

Pawel

ksinkel
Posts: 593
Joined: April 2nd, 2009, 11:58 am
Contact:

Re: Out of memory error processing big files

Post by ksinkel »

For a support problem, please send as detailed a description as possible for support@dl-c.com.

Thank you,

Kiril
Kiril Sinkel
Digital Light & Color

DesertBighorn
Posts: 7
Joined: May 12th, 2009, 10:36 pm

Re: Out of memory error processing big files

Post by DesertBighorn »

Any update on a 64 bit version of PWP? Or did I miss a new release that covers this? :-)

Thanks,
Doug

ksinkel
Posts: 593
Joined: April 2nd, 2009, 11:58 am
Contact:

Re: Out of memory error processing big files

Post by ksinkel »

We have folded the 64 bit version into PWP 6.0. 6.0 will be coming out later this year. As always, we will announce a beta here on the message board when it is ready.

Kiril
Kiril Sinkel
Digital Light & Color

alain
Posts: 95
Joined: September 27th, 2010, 7:18 am
What is the make/model of your primary camera?: A850

Re: Out of memory error processing big files

Post by alain »

Hi

Will the 6.0 version support more multiple CPU calculations?

And a specific question : I'm probably upgrading to either a 4 or 6 core CPU, will 6.0 use 6 cores for some applications?

ksinkel
Posts: 593
Joined: April 2nd, 2009, 11:58 am
Contact:

Re: Out of memory error processing big files

Post by ksinkel »

The number of cores used at any given time is under operating system control. The operating system schedules any available threads to available cores as it sees fit. In the next version, PWP will use multiple threads for some file I/O, workflow and some transformations. In addition, other things running on your computer (anti-virus applications are often big CPU users) also use separate threads so these will also use additional cores.

Kiril
Kiril Sinkel
Digital Light & Color

alain
Posts: 95
Joined: September 27th, 2010, 7:18 am
What is the make/model of your primary camera?: A850

Re: Out of memory error processing big files

Post by alain »

Hi Kiril,

Thanks for the answer, but it's the application that's starting (and keeping "feeding") the threads. Some launch as much threads as CPU's and divide the image in lot's off tiles (for example 128*128 -or 190*190 in size depending on type), if done carefully this can lead to an near linear speed increase if the transformation is complex.

Can you already give a hint if 6 cores will be used (if OS is permitting) when those are available for the transformations?

alain
Posts: 95
Joined: September 27th, 2010, 7:18 am
What is the make/model of your primary camera?: A850

Re: Out of memory error processing big files

Post by alain »

Hi

I did some timing : A saturation transform takes 6 seconds on my images (preview is much faster even on rather big previews) and uses only one core.
This is normally almost the only thing happening on my computer then.

If pw 6 would support 4 cores good I would be able to get up to a 6x speed increase if I switch to a modern cpu with 4 cores or even more with a 6core cpu.
Well even if I keep using my current cpu I would get almost a twofold speedup.


BTW. pw starts very fast.

Post Reply