Php photo gallery TWG | JFUploader | TWG Flash upload | WFU | Forum
https://www.tinywebgallery.com/forum/

Huge Memory Spike and Directory Quota Error for JFU on Linux
https://www.tinywebgallery.com/forum/viewtopic.php?f=13&t=3130
Page 1 of 1

Author:  cknight [ 19. Oct 2011, 20:22 ]
Post subject:  Huge Memory Spike and Directory Quota Error for JFU on Linux

Hi Michael,

I purchased the Enterprise Edition on August 29th, 2011.

Exact JFU Version: 2.14.3
Joomla Version: 1.5.23
Server Type: Linux running CentOS-5.6 Continuous Release i386 and x86_64
TFU Log File Embedded Below
JFU Configuration:
1 Frontend Master Profile with 4 users for business partners
PLUS 1 Frontend (non-Master) Profile for Staff Members landing in the parent directory of the business partner directories ... similar setup as in your video tutorial.

Mostly it is working well, however there are two significant issues we are encountering:

ISSUE #1: Huge Memory Spike During Upload

We were using JFU to upload large video files (up to 1.5 G) with the queue set to a max of 5G. When the client uploaded a set of nearly 5G worth of files in the queue (approx 7 files), there was a HUGE memory spike on our server which used 98% of the total of our 8G of RAM plus our 2G of swap space. Why is JFU chewing up so much memory? -- BTW, I have since set the queue limit down to 1.5G so the client won't crash their server.

ISSUE #2: Erroneous Directory Quota Error Message During Upload Attempt

When attempting to upload files that DO NOT exceed the directory quota limit specified in the JFU configuration for the profile, JFU gives an erroneous 'directory quota exceeded error' and will not allow the user to upload. HOWEVER, if the user clicks on a file in the REMOTE window after the error message appears, it will then allow the user to upload. Very strange error.

Here is the exact error message: "The quota for the directory is 9765MB, Please remove some files." However, this error appears regardless of what the directory quota is set to in the backend.

Please let me know what other information you need.

Thanks,
Claire

10.13.2011 16:43:28 - Action:dir; Directory: ./__/__/__/__/images/file_transfer
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=dir&dir=&ts=1318538673583&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&tfu_ut=-1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:dir; Directory: ./../../../../images/file_transfer
10.13.2011 16:43:57 - Action:dir; Directory: ./__/__/__/__/images/file_transfer
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=dir&changedir=true&dir=&index=1&ts=1318538703248&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:dir; Directory: ./../../../../images/file_transfer
10.13.2011 16:44:21 - Action:info; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=info&dir=&index=0&ts=1318538727135&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:info; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:44:22 - Action:preview; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=preview&index=0&ts=1318538727326&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:preview; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:45:24 - 1. Config loaded
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:48:tfu_debug:1. Config loaded
10.13.2011 16:45:24 - 2. Authenification sucessfull
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:63:tfu_debug:2. Authenification sucessfull
10.13.2011 16:45:24 - 3. Directory read: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:65:tfu_debug:3. Directory read: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:45:24 - 3a. $_FILES content:
Array
(
[Filedata] => Array
(
[name] => test.zip
[type] => application/octet-stream
[tmp_name] => /home/globalreef/tmp/phpJVnWxC
[error] => 0
[size] => 1048576
)

)

Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:78:tfu_debug:3a. $_FILES content:
Array
(
[Filedata] => Array
(
[name] => test.zip

10.13.2011 16:45:24 - 4. Check valid extension: test.zip
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:89:tfu_debug:4. Check valid extension: test.zip
10.13.2011 16:45:24 - 4a. Extension valid.
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:91:tfu_debug:4a. Extension valid.
10.13.2011 16:45:24 - 5. Before move_uploaded_file : /home/globalreef/tmp/phpJVnWxC -> ./__/__/__/__/images/file_transfer/lof_filexfer_bp2/test.zip
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:115:tfu_debug:5. Before move_uploaded_file : /home/globalreef/tmp/phpJVnWxC -> ./../../../../images/file_transfer/
10.13.2011 16:45:24 - 5a. After move_uploaded_file.
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:127:tfu_debug:5a. After move_uploaded_file.
10.13.2011 16:45:24 - 6. Uploaded.
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:148:tfu_debug:6. Uploaded.
10.13.2011 16:45:24 - 7. Internal processing done.
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:181:tfu_debug:7. Internal processing done.
10.13.2011 16:45:24 - 9. End upload
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:254:tfu_debug:9. End upload
10.13.2011 16:45:24 - 10. End store session
Request: /administrator/components/com_jfuploader/tfu/tfu_upload.php?dir=&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0&PHPSESSID=ajjksgmebndihiqfpjtmjkfse0&size=100000&firstStart=true&zeit=1318538764818&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&remaining=1
Stack: tfu_upload.php:256:tfu_debug:10. End store session
10.13.2011 16:45:24 - Action:dir; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=dir&dir=&ts=1318538789390&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&tfu_ut=5&check_upload=true&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:dir; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:48:28 - Action:ping; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=ping&ts=1318538973583&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:ping; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:41 - Action:info; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=info&dir=&index=0&ts=1318539226898&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:info; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:41 - Action:preview; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=preview&index=0&ts=1318539227062&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:preview; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:43 - Action:info; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=info&dir=&index=1&ts=1318539228924&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:info; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:43 - Action:preview; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=preview&index=1&ts=1318539229075&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:preview; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:54 - Action:xdelete; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=xdelete&index=&offset=1&dir=&ts=1318539239403&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:xdelete; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:54 - ERROR 2 in onents/com_jfuploader/tfu/tfu_helper.php, line 1846: Invalid argument supplied for foreach()
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=xdelete&index=&offset=1&dir=&ts=1318539239403&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_helper.php:143:tfu_debug:ERROR 2 in onents/com_jfuploader/tfu/tfu_helper.php, line 1846: Invalid argument supplied for foreac
Stack: tfu_helper.php:1846:on_error:2:Invalid argument supplied for foreach():/home/globalreef/public_html/administrator/components/com_jfuploader/tfu/tfu_helper.php:1846:Array( [file] => [show_delete] => true [deleted] => 0 [perm] => 0 [notdel] => 0)
Stack: tfu_file.php:91:tfu_delete_files::true
10.13.2011 16:52:54 - Action:dir; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=dir&dir=&ts=1318539239496&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&tfu_ut=0&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:dir; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:56 - Action:info; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=info&dir=&index=1&ts=1318539242155&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:info; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:52:57 - Action:preview; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=preview&index=1&ts=1318539242251&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:preview; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:53:01 - Action:delete; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=delete&index=1&dir=&ts=1318539246964&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:delete; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2
10.13.2011 16:53:01 - Action:dir; Directory: ./__/__/__/__/images/file_transfer/lof_filexfer_bp2
Request: /administrator/components/com_jfuploader/tfu/tfu_file.php?action=dir&dir=&ts=1318539247056&tfu_rn=1317390500867326182078535330541&tfu_is_flash_10=1&tfu_ut=0&TFUSESSID=ajjksgmebndihiqfpjtmjkfse0
Stack: tfu_file.php:57:tfu_debug:Action:dir; Directory: ./../../../../images/file_transfer/lof_filexfer_bp2

Author:  TinyWebGallery [ 20. Oct 2011, 09:45 ]
Post subject:  Re: Huge Memory Spike and Directory Quota Error for JFU on L

Hi,

1. JFU uses the normal php functions for the upload. So if this happens during upload JFU has nothing to do with it. After the upload the tfu_upload.php is called and the file is moved from the temp to the normal directory.
So if this process uses a lot of memory then unfortunately this is something php releated I cannot do anything.

For more details the memory usage would have to be dumped in the upload file. This would make sure where the problem is.

2. I have to look at this. Maybe the setting is not mapped properly in the JFU config.

- Michael

Author:  TinyWebGallery [ 21. Oct 2011, 00:46 ]
Post subject:  Re: Huge Memory Spike and Directory Quota Error for JFU on L

2. I have checked. By clicking on the server window the buttons are reseted but the limit is here not checked again.

I have already fixed this in TFU 2.15 an this will be then fixed in JFU 2.15 as well.
For a quick fix you would have to download TFU and patch JFU by yourself by overwriting all files in the tfu folder and renaming the flash to version 2.14.

In this version also a memory dump is available for each request if you enable the enhanced debug. there you can check if the memory spike

Best,
Michael

Page 1 of 1 All times are UTC + 1 hour [ DST ]
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/