0003267: rrdfix.sh not found problem and high cpu load

(0005039)
ra-endian (administrator)
2010-11-08 10:30

I think the rrdfix script should not run every 5 minutes. I check this now.

(0005041)
ardit-endian (developer)
2010-11-08 13:47
edited on: 2010-11-08 13:47

In deed it is in cron.cyclic and from crontab, and run every 5 min:
&nolog */5 * * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.cyclic

cron.cyclic :
# cat /etc/cron.cyclic/makegraphs
#!/bin/sh

[ -x /usr/local/bin/rrdfix.sh ] && /usr/local/bin/rrdfix.sh >/dev/null
/usr/local/bin/makegraphs.pl >/dev/null
exit 0

(0005093)
ra-endian (administrator)
2010-11-11 17:03

ok. you are right.Then for now we should exclude the ntop directory. It's to big for check that directory every 5 minutes.

(0005189)
ardit-endian (developer)
2010-11-24 09:14

Hello ra,

I'm looking now to a system, and it gaves me :

rrdfix.sh
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory

And if we remove the /var/ntop dir from the script then the files under that directory will be not deleted.
i executed the script and under the /var/ntop directory the file count is the same:
# for i in ntop; do echo -n "$i: "; find $i -type f | wc -l; done
ntop: 8201
i suppose that after the error, the script don't check that directory any more..
-------------
Other update, i found the problem i think..
indeed there is no /var/ntop/rrd/flows/Host directory but is:

root@endian:/var/ntop/rrd/flows # cd Host\ Last\ Seen/
root@endian:/var/ntop/rrd/flows/Host Last Seen #

# ls -acl
total 16
drwx------ 4 ntop ntop 4096 Nov 15 17:02 .
drwxr-xr-x 5 ntop ntop 4096 Mar 16 2010 ..
drwx------ 2 ntop ntop 4096 Nov 15 17:02 Host Last Seen
drwx------ 2 ntop ntop 4096 Nov 15 17:02 ICMP Watch

i think the problem will be the same with ICMP Watch also because the directory name is splitted.

the solution would be modifying the script to be able to check directories like /var/ntop/rrd/flows/Host Last Seen # or ntop creating 1 named directories..

Regards,
ardit.

(0005205)
lorenzo-endian (manager)
2010-11-25 11:01

hi ardit,

i have done some tests and on my machine this problem never arises.

on which hardware the software is running?

thanks a lot!

lo

(0005206)
ardit-endian (developer)
2010-11-25 12:06

hi lo :)
please check the mail,

thanks,
ardit

(0005220)
ardit-endian (developer)
2010-11-26 16:30

hi lo,
I also verified on some systems (specially mini) that when ntop is on the CPU shows 100% almost all the time, and i think there is some impact when the script is automatically executed then it runs on the memory all the time..

You can check this thread on efw-support for more info, because different users reported this with the community edition as well.
http://efwsupport.com/index.php?topic=1341.0 [^]

regards
ardit.

(0005262)
lorenzo-endian (manager)
2010-12-02 09:15

Hello ardit,

this problem has been fixed in the last update :)

let me know if the problem persists!

have a nice day

Lo

(0005751)
ardit-endian (developer)
2011-02-24 09:52

root@biofw:~ # rrdfix.sh
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory
ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory

another system

(0005752)
lorenzo-endian (manager)
2011-02-24 10:02

Hi ardit!

as said, the problem exists and we are sure about that.

Now, I will work hard in order to reproduce it, because up to now it arises in an unpredictable way.

Thanks for testing & happy hacking

Lo

(0005968)
lorenzo-endian (manager)
2011-03-16 09:45

We have to release the fix for 2.3 :)

(0008095)
luke-endian (developer)
2012-09-07 14:33

Hi Guys

i've the same problem on a 2.5 x86
i see that when the rrdfix cron in /etc/cron.cyclic starts CPU shows 100% almost all the time.
I have notice that the problem is NTOP folder (/var/ntop)that is 1.4 GB and rrdfix.sh script uses about 4 minutes to be completed and coronjob start every 5 minutes.

(0008096)
luke-endian (developer)
2012-09-07 14:36

Sorry, i would add that i have no "file not found" problem but "only" CPU high workload problem.

Issue History
Date Modified	Username	Field	Change
2010-11-08 09:20	ardit-endian	New Issue
2010-11-08 10:15	luca-endian	Tag Attached: purple
2010-11-08 10:30	ra-endian	Note Added: 0005039
2010-11-08 13:47	ardit-endian	Note Added: 0005041
2010-11-08 13:47	ardit-endian	Note Edited: 0005041
2010-11-11 17:03	ra-endian	Note Added: 0005093
2010-11-24 09:14	ardit-endian	Note Added: 0005189
2010-11-25 11:01	lorenzo-endian	Note Added: 0005205
2010-11-25 11:01	lorenzo-endian	Assigned To	=> lorenzo-endian
2010-11-25 11:01	lorenzo-endian	Status	new => feedback
2010-11-25 12:06	ardit-endian	Note Added: 0005206
2010-11-26 16:30	ardit-endian	Note Added: 0005220
2010-12-02 09:15	lorenzo-endian	Note Added: 0005262
2010-12-02 09:15	lorenzo-endian	Status	feedback => closed
2010-12-02 09:15	lorenzo-endian	Resolution	open => fixed
2010-12-02 09:15	lorenzo-endian	Fixed in Version	=> 2.4.1
2011-01-26 09:19	ardit-endian	Status	closed => feedback
2011-01-26 09:19	ardit-endian	Resolution	fixed => reopened
2011-02-15 10:53	lorenzo-endian	Customer Occurencies	=> 0
2011-02-15 10:53	lorenzo-endian	Assigned To	lorenzo-endian => ardit-endian
2011-02-24 09:52	ardit-endian	Note Added: 0005751
2011-02-24 10:02	lorenzo-endian	Note Added: 0005752
2011-02-24 10:02	lorenzo-endian	Assigned To	ardit-endian => lorenzo-endian
2011-02-24 10:02	lorenzo-endian	Status	feedback => confirmed
2011-03-16 09:45	lorenzo-endian	Note Added: 0005968
2011-03-16 09:45	lorenzo-endian	Assigned To	lorenzo-endian => peter-endian
2012-09-07 14:33	luke-endian	Note Added: 0008095
2012-09-07 14:36	luke-endian	Note Added: 0008096

Notes
(0005039) ra-endian (administrator) 2010-11-08 10:30	I think the rrdfix script should not run every 5 minutes. I check this now.

(0005041) ardit-endian (developer) 2010-11-08 13:47 edited on: 2010-11-08 13:47	In deed it is in cron.cyclic and from crontab, and run every 5 min: &nolog /5 * * * [ -x /bin/run-parts ] && run-parts --report /etc/cron.cyclic cron.cyclic : # cat /etc/cron.cyclic/makegraphs #!/bin/sh [ -x /usr/local/bin/rrdfix.sh ] && /usr/local/bin/rrdfix.sh >/dev/null /usr/local/bin/makegraphs.pl >/dev/null exit 0

(0005093) ra-endian (administrator) 2010-11-11 17:03	ok. you are right.Then for now we should exclude the ntop directory. It's to big for check that directory every 5 minutes.

(0005189) ardit-endian (developer) 2010-11-24 09:14	Hello ra, I'm looking now to a system, and it gaves me : rrdfix.sh ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory And if we remove the /var/ntop dir from the script then the files under that directory will be not deleted. i executed the script and under the /var/ntop directory the file count is the same: # for i in ntop; do echo -n "$i: "; find $i -type f \| wc -l; done ntop: 8201 i suppose that after the error, the script don't check that directory any more.. ------------- Other update, i found the problem i think.. indeed there is no /var/ntop/rrd/flows/Host directory but is: root@endian:/var/ntop/rrd/flows # cd Host\ Last\ Seen/ root@endian:/var/ntop/rrd/flows/Host Last Seen # # ls -acl total 16 drwx------ 4 ntop ntop 4096 Nov 15 17:02 . drwxr-xr-x 5 ntop ntop 4096 Mar 16 2010 .. drwx------ 2 ntop ntop 4096 Nov 15 17:02 Host Last Seen drwx------ 2 ntop ntop 4096 Nov 15 17:02 ICMP Watch i think the problem will be the same with ICMP Watch also because the directory name is splitted. the solution would be modifying the script to be able to check directories like /var/ntop/rrd/flows/Host Last Seen # or ntop creating 1 named directories.. Regards, ardit.

(0005205) lorenzo-endian (manager) 2010-11-25 11:01	hi ardit, i have done some tests and on my machine this problem never arises. on which hardware the software is running? thanks a lot! lo

(0005206) ardit-endian (developer) 2010-11-25 12:06	hi lo :) please check the mail, thanks, ardit

(0005220) ardit-endian (developer) 2010-11-26 16:30	hi lo, I also verified on some systems (specially mini) that when ntop is on the CPU shows 100% almost all the time, and i think there is some impact when the script is automatically executed then it runs on the memory all the time.. You can check this thread on efw-support for more info, because different users reported this with the community edition as well. http://efwsupport.com/index.php?topic=1341.0 [^] regards ardit.

(0005262) lorenzo-endian (manager) 2010-12-02 09:15	Hello ardit, this problem has been fixed in the last update :) let me know if the problem persists! have a nice day Lo

(0005751) ardit-endian (developer) 2011-02-24 09:52	root@biofw:~ # rrdfix.sh ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory ERROR: opening '/var/ntop/rrd/flows/Host': No such file or directory another system

(0005752) lorenzo-endian (manager) 2011-02-24 10:02	Hi ardit! as said, the problem exists and we are sure about that. Now, I will work hard in order to reproduce it, because up to now it arises in an unpredictable way. Thanks for testing & happy hacking Lo

(0005968) lorenzo-endian (manager) 2011-03-16 09:45	We have to release the fix for 2.3 :)

(0008095) luke-endian (developer) 2012-09-07 14:33	Hi Guys i've the same problem on a 2.5 x86 i see that when the rrdfix cron in /etc/cron.cyclic starts CPU shows 100% almost all the time. I have notice that the problem is NTOP folder (/var/ntop)that is 1.4 GB and rrdfix.sh script uses about 4 minutes to be completed and coronjob start every 5 minutes.

(0008096) luke-endian (developer) 2012-09-07 14:36	Sorry, i would add that i have no "file not found" problem but "only" CPU high workload problem.

Relationships

Please see now our new Bugtracker system: JIRA