SYSTEM WARNING: 'date_default_timezone_get(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone.' in '/usr/share/mantis/www/core.php' line 264

0003332: Proxy graphs not working - MantisBT
MantisBT - Endian Firewall
View Issue Details
0003332Endian FirewallGUIpublic2010-11-26 23:172010-12-09 12:01
gmar_87 
max-endian 
normalminoralways
resolvedfixed 
 
 
0003332: Proxy graphs not working
Proxy graphs aren't displayed in 2.4.1. Just displays "No information available. "
Happens on 3 different installs on 3 different hardware spec servers.
2 installs used config restored from 2.3 and 1 install had config restored from 2.4.1.
/var/log/httpd/error.log
************************
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.

No tags attached.
7z Access_Logs.7z (258,344) 2010-12-07 11:38
https://bugs.endian.com/file_download.php?file_id=564&type=bug
? squid.conf (7,252) 2010-12-08 09:27
https://bugs.endian.com/file_download.php?file_id=566&type=bug
jpg system_graphs.jpg (21,117) 2010-12-09 02:22
https://bugs.endian.com/file_download.php?file_id=572&type=bug
jpg

jpg traffic_graphs.jpg (30,609) 2010-12-09 02:23
https://bugs.endian.com/file_download.php?file_id=573&type=bug
jpg

? header.pl (62,921) 2010-12-09 08:47
https://bugs.endian.com/file_download.php?file_id=574&type=bug
? makegraphs.pl (17,643) 2010-12-09 08:47
https://bugs.endian.com/file_download.php?file_id=575&type=bug
Issue History
2010-11-26 23:17gmar_87New Issue
2010-11-28 20:32ra-endianAssigned To => max-endian
2010-12-06 17:34max-endianNote Added: 0005308
2010-12-06 17:34max-endianStatusnew => feedback
2010-12-07 02:11pwizardNote Added: 0005309
2010-12-07 09:00gmar_87Note Added: 0005312
2010-12-07 10:29max-endianNote Added: 0005314
2010-12-07 11:38gmar_87File Added: Access_Logs.7z
2010-12-07 11:42gmar_87Note Added: 0005316
2010-12-07 13:31max-endianNote Added: 0005317
2010-12-08 09:27gmar_87File Added: squid.conf
2010-12-08 09:27gmar_87Note Added: 0005319
2010-12-08 09:54gmar_87Note Added: 0005320
2010-12-08 10:13max-endianNote Added: 0005321
2010-12-08 10:15max-endianNote Added: 0005322
2010-12-08 10:39gmar_87Note Added: 0005323
2010-12-08 13:12max-endianNote Added: 0005324
2010-12-09 02:22pwizardFile Added: system_graphs.jpg
2010-12-09 02:22pwizardNote Added: 0005328
2010-12-09 02:23pwizardFile Added: traffic_graphs.jpg
2010-12-09 08:47max-endianFile Added: header.pl
2010-12-09 08:47max-endianFile Added: makegraphs.pl
2010-12-09 08:51max-endianNote Added: 0005334
2010-12-09 10:01pwizardNote Added: 0005338
2010-12-09 10:54max-endianNote Added: 0005339
2010-12-09 10:54max-endianStatusfeedback => resolved
2010-12-09 10:54max-endianResolutionopen => fixed
2010-12-09 11:23gmar_87Note Added: 0005341
2010-12-09 11:23gmar_87Statusresolved => feedback
2010-12-09 11:23gmar_87Resolutionfixed => reopened
2010-12-09 12:01max-endianNote Added: 0005342
2010-12-09 12:01max-endianStatusfeedback => resolved
2010-12-09 12:01max-endianResolutionreopened => fixed

Notes
(0005308)
max-endian   
2010-12-06 17:34   
Hi!
I am not able to reproduce the problem. It would be a problem, if you could send me the your proxy logs? You can replace all private data with 'x' or something I'm only intrested in the numbers! ;)

max
(0005309)
pwizard   
2010-12-07 02:11   
Hi
refer to http://bugs.endian.it/view.php?id=3333 [^]
please change /var/lib/python2.4/site-packages/endian/emi/
to /usr/lib/python2.4/site-packages/endian/emi/
(0005312)
gmar_87   
2010-12-07 09:00   
@max, can you please provide names of logs that your require?

@pwizard, I have applied the fix for issue 3333, but proxy graphs still aren't being displayed.

here is tail -n 200 /var/log/httpd

Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/wpad.dat line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/wpad.dat line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/wpad.dat line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/wpad.dat line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 154.
Use of uninitialized value in substitution (s///) at /home/httpd/html/proxy.pac line 157.
[Tue Dec 7 19:30:10 2010] [error] [client 10.1.1.2] File does not exist: /home/httpd/html/include/jquery-calendar.css
Modification of a read-only value attempted at /home/httpd/cgi-bin/logs_proxy.cgi line 416.
[Tue Dec 7 19:30:19 2010] [error] [client 10.1.1.2] File does not exist: /home/httpd/html/include/jquery-calendar.css
(0005314)
max-endian   
2010-12-07 10:29   
Hi!
I would need /var/log/squid/access.log_short and /var/log/squid/access.log_shortXXXXX where XXXXX would be the date of the last day.
Thank you!
(0005316)
gmar_87   
2010-12-07 11:42   
Hi Max,

I exported and modified logs you requested, but was unable to upload them as *.XLS files (97-2003 Excel Spreadsheet) - I receive "APPLICATION ERROR #401". Are XLS files prohibited from uploads?
 
I added them to 7-zip archive and upload worked.

Regards,
John
(0005317)
max-endian   
2010-12-07 13:31   
Hi,
thanks!

I saved your files on my fw and I couldn't create anymore the graphs. But your file has less entries then my old one. For example this is a line of my entry:

1291315802.068 0 192.168.1.2 TCP_DENIED/403 2760 GET http://flowplayer.org/favicon.ico [^] - NONE/- text/html

In your lines are missing the last 5 entries(the address you probably have removed, so I added dummy one..). Is this correct or I made mistake in the conversion?

In your /etc/squid/squid.conf is a setting "logformat"?
(0005319)
gmar_87   
2010-12-08 09:27   
Hi,

The first line in access.log_short-20101206 looks like this:
1291555721.118 785 127.0.0.1 TCP_MISS/304 267 GET http://um10.eset.com/eset_upd/update.ver [^] - DIRECT/93.184.71.21 -

I removed everything after the address for the spread sheet...

I have attached my squid.conf
(0005320)
gmar_87   
2010-12-08 09:54   
I found this in /usr/local/bin/makegraphs.pl
Is it correct?

### Squid Graphs
###
if (fileStayInMemory("/var/log/squid/access.log_short")) {
        system ('cat /var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d) /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgrap
h --tcp-only >/dev/null 2>&1');
}

###
(0005321)
max-endian   
2010-12-08 10:13   
Ok, it seems like squid is logging correctly.

Your /home/httpd/html/sgraph(sgraph not graphs) contains something? There should be those files:
index.html tcp-access.png tcp-duration.png tcp-transfer.png udp-access.png udp-duration.png udp-transfer.png

Yes, this is "correct". But maybe you have the same bug like I had. What does fileStayInMemory return?

Here are the bugfixes:

http://bugs.endian.com/view.php?id=3351 [^]

and the secend one is more optimising, but it should not have an affect, if the access.log_short of the previous day exists:

Replace in the file /usr/local/bin/makegraph.pl the section Squid Graphs by this one:

###
### Squid Graphs
###
if (fileStayInMemory("/var/log/squid/access.log_short")) {
if(-e '/var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d)' ){
        system ('cat /var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d) /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/http$
}else{
        system ('cat /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only >/dev/null 2>&1');
}

}
(0005322)
max-endian   
2010-12-08 10:15   
The link to the second bugfix:

http://bugs.endian.com/view.php?id=3350 [^]
(0005323)
gmar_87   
2010-12-08 10:39   
/home/httpd/html/sgraph is empty.

Replaced Squid graph section of makegraphs.pl with what you supplied in previous, but now see this when manually executing makegraphs.pl

Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "system ('cat"
  (Might be a runaway multi-line '' string starting on line 473)
        (Do you need to predeclare system?)
Warning: Use of "log" without parentheses is ambiguous at /usr/local/bin/makegraphs.pl line 475.
Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "/squid/access"
        (Missing operator before access?)
Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "/usr/bin"
        (Missing operator before bin?)
Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "/home/httpd"
        (Missing operator before httpd?)
Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "--tcp"
        (Missing operator before tcp?)
Bareword found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "/dev/null"
        (Missing operator before null?)
Number found where operator expected at /usr/local/bin/makegraphs.pl line 475, near "null 2"
        (Do you need to predeclare null?)
String found where operator expected at /usr/local/bin/makegraphs.pl line 475, at end of line
        (Missing semicolon on previous line?)
syntax error at /usr/local/bin/makegraphs.pl line 475, near "system ('cat "
Can't find string terminator "'" anywhere before EOF at /usr/local/bin/makegraphs.pl line 475.

After this no graphs are shown in the GUI. I commented out the squid graphs section and i get this.

libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error
libpng error: Read Error

All graphs except proxy are displayed in the GUI.

>>What does fileStayInMemory return?
How do i check this? I received:
-bash: fileStayInMemory: command not found

Cheers,
John
(0005324)
max-endian   
2010-12-08 13:12   
Oh sorry, during the there is an error in the code above which probably causes the error above:

###
### Squid Graphs
###
if (fileStayInMemory("/var/log/squid/access.log_short")) {
if(-e '/var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d)' ){
        system ('cat /var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d) /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only >/dev/null 2>&1');
}else{
        system ('cat /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only >/dev/null 2>&1');
}

}

This should work(Sorry, I can not try it...).

But you can try to execute this(in the bash), this is the commando which should create the proxy graphs:

cat /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only


fileStayInMemory is a perl function of the fw. But if you have no perl experiance let it be.

Max
(0005328)
pwizard   
2010-12-09 02:22   
After replaced file /usr/local/bin/makegraphs.pl
with

###
### Squid Graphs
###
if (fileStayInMemory("/var/log/squid/access.log_short")) {
if(-e '/var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d)' ){
        system ('cat /var/log/squid/access.log_short-$(date --date "yesterday" +%Y%m%d) /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only >/dev/null 2>&1');
}else{
        system ('cat /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only >/dev/null 2>&1');
}

}

but system graphs and traffic graphs not display.
(0005334)
max-endian   
2010-12-09 08:51   
The replacement in makegraphs.pl removes only the bug, that there is no accesslog of the previous day, no graphs are displayed.

Seen that there are always problems pasting around code, I uploaded you my header.pl and makegraphs.pl, bove are patched. Please save them on the fw at:

header.pl: /var/efw/header.pl
makegraphs.pl: /usr/local/bin/makegraphs.pl

If it is still not working, please let me know, what the following commando does return:

cat /var/log/squid/access.log_short | /usr/bin/squid-graph -o=/home/httpd/html/sgraph --tcp-only

Thanks, max.
(0005338)
pwizard   
2010-12-09 10:01   
Hi,max
Now, both graphs working now.

Many thanks.
(0005339)
max-endian   
2010-12-09 10:54   
Ok, perfect.

Thank you for your feedback!
(0005341)
gmar_87   
2010-12-09 11:23   
Proxy graphs now work - Thanks :)
Traffic and system graphs were reset though...

Will this fix be available via efw-upgrade or will i have to do it manually on all efw installs?

Thanks again.
(0005342)
max-endian   
2010-12-09 12:01   
Hmm I think you have to wait until the next release, because the bug is not a fatal one and occurs not on all systems.

Max.