SYSTEM WARNING: 'date_default_timezone_get(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone.' in '/usr/share/mantis/www/core.php' line 264

0002681: Dashboard stats have wrong values - MantisBT Endian Bugtracker
Endian Issue Tracker





Please see now our new Bugtracker system: JIRA








View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002681Endian FirewallGUIpublic2010-02-13 10:402010-11-22 12:08
Reporterbaldy 
Assigned Topeter-endian 
PriorityhighSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version2.3 
Target Version2.3.1Fixed in Version2.4.1 
Summary0002681: Dashboard stats have wrong values
DescriptionStatistics for the pop3 proxy in the dashboard are way too high.

Stats show for the last hour around 2000 received mails and for Today around 4000.

In reality there should only be about 5 per hour and max 50 per day.

Tagspurple
Attached Files

- Relationships
related to 0002785closedpeter-endian collectd unixsocket sometimes refueses connections 

-  Notes
(0003792)
luca-endian (developer)
2010-02-15 14:20

Same for smtp proxy
(0003818)
baldy (reporter)
2010-02-18 20:44
edited on: 2010-02-19 17:53

Found similar issue in memory usage.

Dashboard shows 26% RAM in use, on the status page shows 38% RAM in use.

Looks like the dashboard shows the +/- buffers/cache value instead of actual ram usage.

Similar to http://bugs.endian.com/view.php?id=2195 [^]

Regards,

Klaas-Jan

(0004039)
ra-endian (administrator)
2010-03-16 08:56

The pop counter on the dashboard are counting connections not e-mails.

logfile:
Mar 15 16:17:25 efw-v2 p3scan[27016]: POP3 Connection from 10.0.0.4:19120
Mar 15 16:17:25 efw-v2 p3scan[27016]: Real-server address is 85.25.144.249:110
Mar 15 16:17:25 efw-v2 p3scan[27016]: USER 'userxy'
Mar 15 16:17:27 efw-v2 p3scan[27016]: Session done (Clean Exit). Mails: 0 Bytes: 0

configuration:
/etc/collectd.d/p3scan.conf
<Plugin "tail">
  <File "/var/log/messages">
    Instance "pop"
    Missingok "on"
    <Match>
      Regex "POP3 Connection from "
      DSType "CounterInc"
      Type "connections"
      Instance "request"
    </Match>
(0004043)
luca-endian (developer)
2010-03-16 10:09

The values however seem too high even for counting the connections
(0004067)
peter-endian (administrator)
2010-03-18 17:14

well, the memory usage percentage on dashboard ans status.cgi differs because dashboard uses memory usage - cache/buffers, which is what really counts.

status.cgi shows both percentages. we should remove that on status.cgi, but that page will disappear anyway in future, so unsure if we really should touch that.
(0004068)
peter-endian (administrator)
2010-03-18 17:19

http proxy hit/miss will be double counted, since the connection passes twice
(squid -> havp -> dansguardian -> squid) which will double-log
(0004069)
peter-endian (administrator)
2010-03-18 19:29

HTTP Proxy 'virus found' counter appears twice, while filter-counter does not show up.
-> Wrong label and will be read out from wrong rrd file
(0004070)
peter-endian (administrator)
2010-03-18 19:32

POP Proxy:
p3scan does not log whether a virus or spam has been found or not, so we can eliminate those counters or need to patch p3scan
(0004073)
peter-endian (administrator)
2010-03-19 17:11

there's no possibility to filter away squid access from 127.0.0.1 :(
i think we need to patch collectd in order to add a ExcludeRegEx
(0004074)
peter-endian (administrator)
2010-03-19 20:42

- fixed pop proxy virus regexp (which was completely wrong)
- changed pop3 counter to count scanned mails instead of connections

spam counter is still missing, needs the p3scan patch

the fetched values are still strange.
rrdtool images show 400M scanned mails when there were only about 1000, and found 800m virus, when it was 1 (ok this could be ok, due to the fact that it is a rate per second not an absolute value)
(0004077)
peter-endian (administrator)
2010-03-22 15:37

p3scan counts now also spam mails and displays them in dashboard
(0004078)
peter-endian (administrator)
2010-03-22 15:38

found a new issue: 0002785
(0004079)
peter-endian (administrator)
2010-03-22 16:09

and another one:
dashboard counter are counted *always* twice.

this is easily reproduceable:

logger -p local6.info -t squid <<EOF
Mar 19 15:24:55 UPLINKSTEST2 squid[3189]: 1269008695.976 134 127.0.0.1 \
TCP_MISS/200 1746 GET http://intranet.endian.it/logo.gif [^] - \
DIRECT/80.190.199.145 image/gif
EOF

appears as 1, and then after a short time increases by another 1
same on p3scan counters

i think this is on every other counter, too


probably this is because we read multiple times witin the collectd 5 seconds interval where the data does not change.
so if already read out the same value will be read out a second time.

probably we need to check if the timestamp of the last read is older than the current timestamp and ignore if it is (?)
(0004082)
peter-endian (administrator)
2010-03-22 22:37

values counting twice is fixed
(0004083)
peter-endian (administrator)
2010-03-22 23:10

dansguardian counter is correct in collectd, but is always 0 on gui
(0004084)
peter-endian (administrator)
2010-03-22 23:43

dansguardian is ok now.

- http proxy values are all ok, last missing is "miss" which is counted twice (needs collectd patch)
- pop3 proxy values are also all ok now

smtp proxy:
- mails in queue is ok
- clean mail received is ok
- viruses found is ok
- mails received counts twice (from client and from amavis) (needs collectd
  patch)
- mails blocked does work but is in fact mails rejected.. it is not virus
  or spam mails blocked -> rename it
(0004089)
peter-endian (administrator)
2010-03-23 20:29

- collectd patch is now ready
- smtp and proxy counters are ok now

last open things:
- collectd sometimes does not remove the socket file
- values sometimes are completely wrong (overflow?)
(0004101)
peter-endian (administrator)
2010-03-26 15:15

- unixsock patch is sent upstream
- values have now an upper limit as suggested by collectd mailinglist

- Issue History
Date Modified Username Field Change
2010-02-13 10:40 baldy New Issue
2010-02-15 09:03 luca-endian Status new => confirmed
2010-02-15 14:20 luca-endian Note Added: 0003792
2010-02-15 14:23 luca-endian Summary POP3 proxy stats way too high => Dashboard stats have wrong values
2010-02-15 14:23 luca-endian Tag Attached: purple
2010-02-18 20:44 baldy Note Added: 0003818
2010-02-18 20:47 baldy Note Edited: 0003818
2010-02-19 17:53 baldy Note Edited: 0003818
2010-03-03 16:01 ra-endian Assigned To => ra-endian
2010-03-03 16:01 ra-endian Status confirmed => assigned
2010-03-03 16:01 ra-endian Target Version => future
2010-03-04 07:56 ra-endian Priority normal => high
2010-03-04 07:56 ra-endian Target Version future => 2.3.1
2010-03-15 15:06 ra-endian Severity tweak => major
2010-03-16 08:56 ra-endian Note Added: 0004039
2010-03-16 10:09 luca-endian Note Added: 0004043
2010-03-18 08:07 ra-endian Status assigned => new
2010-03-18 08:07 ra-endian Assigned To ra-endian => peter-endian
2010-03-18 08:07 ra-endian Status new => confirmed
2010-03-18 17:14 peter-endian Note Added: 0004067
2010-03-18 17:19 peter-endian Note Added: 0004068
2010-03-18 19:29 peter-endian Note Added: 0004069
2010-03-18 19:32 peter-endian Note Added: 0004070
2010-03-19 17:11 peter-endian Note Added: 0004073
2010-03-19 20:42 peter-endian Note Added: 0004074
2010-03-22 15:37 peter-endian Note Added: 0004077
2010-03-22 15:38 peter-endian Note Added: 0004078
2010-03-22 15:38 peter-endian Relationship added related to 0002785
2010-03-22 16:09 peter-endian Note Added: 0004079
2010-03-22 22:37 peter-endian Note Added: 0004082
2010-03-22 23:10 peter-endian Note Added: 0004083
2010-03-22 23:43 peter-endian Note Added: 0004084
2010-03-23 20:29 peter-endian Note Added: 0004089
2010-03-26 15:14 peter-endian Status confirmed => resolved
2010-03-26 15:14 peter-endian Fixed in Version => 2.3.1
2010-03-26 15:14 peter-endian Resolution open => fixed
2010-03-26 15:15 peter-endian Note Added: 0004101
2010-11-22 12:08 peter-endian Fixed in Version 2.3.1 => 2.4.1
2010-11-22 12:08 peter-endian Status resolved => closed

Copyright © 2005-2008 Endian, SRL. All rights reserved.


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker