SYSTEM WARNING: 'date_default_timezone_get(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone.' in '/usr/share/mantis/www/core.php' line 264

0002684: Manual whitelist not processed properly - MantisBT Endian Bugtracker
Endian Issue Tracker





Please see now our new Bugtracker system: JIRA







Main | My View | View Issues | Change Log | Roadmap
  

View Issue Details [ Jump to Notes ] [ Issue History ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002684Endian FirewallProxy SMTPpublic2010-02-15 12:242010-11-22 12:08
Reporterbaldy 
Assigned Tosimon-endian 
PriorityhighSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version2.3 
Target Version2.3.1Fixed in Version2.4.1 
Summary0002684: Manual whitelist not processed properly
DescriptionEmail addresses from the manual whitelist in the SMTP proxy configuration should get an AWL value of -100.
Additional InformationThis does not happen so whitelisted addresses still end up in spam-quarantine.

In the mailheaders there is no AWL value -100, just a value between -2 and +2, which indicates normal processing by the auto whitelist, and not the manual whitelist.

Regards,

Klaas-Jan
TagsNo tags attached.
Attached Filespatch file icon spamassassin_manual_lists.patch [^] (1,847 bytes) 2010-02-22 18:15 [Show Content]
? file icon restartsmtpscan.py [^] (20,545 bytes) 2010-02-23 09:16
? file icon local.cf.tmpl [^] (464 bytes) 2010-02-23 09:17

- Relationships
has duplicate 0002655closed smtp proxy whitelist don't work for some kind of emails? 

-  Notes
(0003794)
peter-endian (administrator)
2010-02-15 15:52

 manual whitelisting has nothing to do with AWL!

read here:
http://wiki.apache.org/spamassassin/ManualWhitelist [^]
http://wiki.apache.org/spamassassin/AutoWhitelist [^]

manual whitelisted from's get a score value of -100, not an AWL.
AWL does not know anything about your manual whitelists

The name of AWL (auto-white-listing) as spamassassin calls it, is ambiguous. In fact, AWL is an automatic score averaging system and has nothing to do with whitelisting!
(0003795)
peter-endian (administrator)
2010-02-15 15:54

 Please report back if you see that -100 score (not AWL) in header. of better, post your headers and your /etc/spamassassin/local.cf
(0003797)
baldy (reporter)
2010-02-15 16:16

 Peter,

Sorry for incorrectly frasing my question and confusing the AWL mechanism with manual whitelisting, this is also why I incorrectly reported issue 2680.
I thought that the -100 would be an AWL value in the headers.

The -100 score is never shown in the headers.

local.cf also does not show the manual whitelisted addresses.

When adding new emailaddresses to the manual whitelist in the gui the local.cf file is touched, date and time changes, but is not updated.

Local.cf

# amavis spamassassin config file

dns_available yes
report_safe 1
use_pyzor 1
pyzor_path /usr/bin/pyzor

use_auto_whitelist 1
use_bayes 1
auto_whitelist_path /var/lib/spamassassin/auto-whitelist
bayes_path /var/lib/spamassassin/bayes

score FORGED_MUA_OUTLOOK 0

Headers for a manual whitelisted address :

X-Spam-Flag: YES
X-Spam-Score: 4.808
X-Spam-Level: ****
X-Spam-Status: Yes, score=4.808 tag=4 tag2=4 kill=4 tests=[AWL=1.194,
    HTML_IMAGE_RATIO_06=0.001, HTML_MESSAGE=0.001, INVALID_DATE=1.651,
    MIME_HTML_MOSTLY=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
    URIBL_BLACK=1.961] autolearn=no
X-Spam-Report: * 2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
 * [URIs: ddma.nl]
 * 1.7 INVALID_DATE Invalid Date: header (not RFC 2822)
 * -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
 * -0.0 SPF_PASS SPF: sender matches SPF record
 * 0.0 HTML_IMAGE_RATIO_06 BODY: HTML has a low ratio of text to image area
 * 0.0 MIME_HTML_MOSTLY BODY: Multipart message mostly text/html MIME
 * 0.0 HTML_MESSAGE BODY: HTML included in message
 * 1.2 AWL AWL: From: address is in the auto white-list
(0003805)
baldy (reporter)
2010-02-16 13:40

 Full headers from address on manual whitelist.
No -100 value found.

Received: from mail.nospam_baldy.nl (192.168.200.1) by remote.baldy.nl
 (192.168.200.4) with Microsoft SMTP Server (TLS) id 8.1.393.1; Tue, 16 Feb
 2010 13:09:36 +0100
Received: from localhost (localhost.localhost [127.0.0.1]) by mail.nospam_baldy.nl
 (Postfix) with ESMTP id 6A449C5A83 for <spam@nospam_baldy.nl>; Tue, 16 Feb 2010
 13:09:36 +0100 (CET)
X-Envelope-From: <mmbounced@nospam_returns.secureserver.net>
X-Envelope-To: <kvdb@nospam_baldy.nl>
X-Envelope-To-Blocked: <kvdb@nospam_baldy.nl>
X-Quarantine-ID: <3g+ylCKD0YWJ>
X-Spam-Flag: YES
X-Spam-Score: 64
X-Spam-Level: ****************************************************************
X-Spam-Status: Yes, score=x+5 tag=4 tag2=4 kill=4 BLACKLISTED
    tests=[AM:BOOST=5] autolearn=unavailable
Received: from mail.nospam_baldy.nl ([127.0.0.1]) by localhost
 (efw-1265401511.localdomain [127.0.0.1]) (amavisd-new, port 10024) with ESMTP
 id 3g+ylCKD0YWJ for <kvdb@nospam_baldy.nl>; Tue, 16 Feb 2010 13:09:36 +0100 (CET)
Received: from m1plcorpmail009.prod.mesa1.secureserver.net
 (m1plcorpmail009.prod.mesa1.secureserver.net [208.109.78.229]) by
 mail.nospam_baldy.nl (Postfix) with SMTP id 5B089C5A82 for <kvdb@nospam_baldy.nl>; Tue, 16
 Feb 2010 13:09:34 +0100 (CET)
Received: (qmail 8899 invoked from network); 16 Feb 2010 12:09:33 -0000
Received: from unknown (HELO m1pwmassmail003.dc1.corp.gd) (208.109.138.146)
  by m1plcorpmail009.prod.mesa1.secureserver.net with SMTP; 16 Feb 2010
 12:09:33 -0000
Received: from mail pickup service by m1pwmassmail003.dc1.corp.gd with
 Microsoft SMTPSVC; Tue, 16 Feb 2010 05:09:33 -0700
X-MID: 8DE3487E-1A58-42C1-ABC3-56267239B578
X-BatchID: 17313
X-10pl8ID: 9381
thread-index: AcqvAJ9DCZjK1CxuQnCcymN3FhdFrg==
Thread-Topic: Take 30% OFF .COM Domains
Reply-To: <offers@nospam_godaddy.com>
From: GoDaddy.com <offers@nospam_godaddy.com>
To: <kvdb@nospam_baldy.nl>
Subject: Take 30% OFF .COM Domains
MIME-Version: 1.0
Content-Type: multipart/alternative;
    boundary="----=_NextPart_000_176194_01CAAEC5.F2E47810"
Content-Class: urn:content-classes:message
Importance: normal
Priority: normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.3959
Message-ID: <M1PWMASSMAIL003HTtt0008d569@m1pwmassmail003.dc1.corp.gd>
X-OriginalArrivalTime: 16 Feb 2010 12:09:33.0270 (UTC) FILETIME=[E6556F60:01CAAF00]
Date: Tue, 16 Feb 2010 05:09:33 -0700
Return-Path: <>
X-MS-Exchange-Organization-Antispam-Report: IPOnAllowList
X-MS-Exchange-Organization-SCL: -1
(0003807)
baldy (reporter)
2010-02-16 14:33

 Other efw 2.3 system with the same problem.

Microsoft Mail Internet Headers Version 2.0
Received: from [192.168.1.64] ([192.168.0.254]) by nospam_goll.nl with Microsoft SMTPSVC(6.0.3790.3959);
     Mon, 15 Feb 2010 13:39:43 +0100
Received: from localhost (localhost.localhost [127.0.0.1])
    by efw-1265455015.localdomain (Postfix) with ESMTP id 5310ADF203
    for <spam@nospam_goll.nl>; Mon, 15 Feb 2010 13:42:43 +0100 (CET)
X-Envelope-From: <bounce_bc@nospam_datagenerator.nl>
X-Envelope-To: <info@nospam_goll.nl>
X-Envelope-To-Blocked: <info@nospam_goll.nl>
X-Quarantine-ID: <FzBKCp3za1gN>
X-Spam-Flag: YES
X-Spam-Score: 5.825
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.825 tag=4 tag2=4 kill=4 tests=[AWL=-1.390,
    FUZZY_AMBIEN=0.962, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.672,
    MIME_HTML_ONLY_MULTI=0.001, MIME_QP_LONG_LINE=1.819,
    MPART_ALT_DIFF=1.143, RCVD_IN_SORBS_WEB=1.117, URI_NOVOWEL=0.5]
    autolearn=no
X-Spam-Report:
 * 1.1 RCVD_IN_SORBS_WEB RBL: SORBS: sender is a abuseable web server
 * [85.92.140.160 listed in dnsbl.sorbs.net]
 * 1.0 FUZZY_AMBIEN BODY: Attempt to obfuscate words in spam
 * 0.5 URI_NOVOWEL URI: URI hostname has long non-vowel sequence
 * 0.0 HTML_MESSAGE BODY: HTML included in message
 * 1.1 MPART_ALT_DIFF BODY: HTML and text parts are different
 * 1.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 * 1.8 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars
 * 0.0 MIME_HTML_ONLY_MULTI Multipart message only has text/html MIME parts
 * -1.4 AWL AWL: From: address is in the auto white-list
Received: from [192.168.1.64] ([127.0.0.1])
    by localhost (efw-1265455015.localdomain [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id FzBKCp3za1gN for <info@nospam_goll.nl>;
    Mon, 15 Feb 2010 13:42:38 +0100 (CET)
Received: from psmtp10.wxs.nl (psmtp10.wxs.nl [195.121.247.26])
    by efw-1265455015.localdomain (Postfix) with ESMTP id BE2FFDF202
    for <info@nospam_goll.nl>; Mon, 15 Feb 2010 13:42:38 +0100 (CET)
Received: from mailserver.businesscompleet.nl (mailserver.businesscompleet.nl [85.92.140.160])
    by psmtp10.wxs.nl (8.13.8/8.12.10) with ESMTP id o1FCgbEb014591
    for <info@nospam_goll.nl>; Mon, 15 Feb 2010 13:42:38 +0100
Received: from s1.businesscompleet.nl ([85.92.144.97] helo=localhost.localdomain)
    by mailserver.businesscompleet.nl with esmtp (Exim 4.69)
    (envelope-from <bounce_bc@datagenerator.nl>)
    id 1Nh0A4-0006qh-2Q
    for info@nospam_goll.nl; Mon, 15 Feb 2010 13:34:48 +0100
To: Koopmans <info@nospam_goll.nl>
From: Redactie Businesscompleet <noreply@businesscompleet.nl>
Reply-To: Redactie Businesscompleet <noreply@businesscompleet.nl>
Subject: Koop eigen risico UWV af * Meer kredietgarantie voor het mkb
Date: Mon, 15 Feb 2010 13:34:49 +0100
X-LibVersion: 3.3.2
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="_=_swift-6262254834b793f692c4fa8.30958995_=_"
Content-Transfer-Encoding: 7bit
Message-ID: <20100215123449.7963.1424564708.swift@srv1a.enilno.nl>
Return-Path: <>
X-OriginalArrivalTime: 15 Feb 2010 12:39:43.0750 (UTC) FILETIME=[F30D0260:01CAAE3B]

--_=_swift-6262254834b793f692c4fa8.30958995_=_
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable


--_=_swift-6262254834b793f692c4fa8.30958995_=_--
(0003810)
peter-endian (administrator)
2010-02-17 14:25

 white/blacklisting configuration in template /etc/spamassassin/local.cf is wrong

an example configuration of:
whitelist:
test@test.com
blah@blah.com
hui@hui.com

blacklist:
black@black.org
black@white.org


produces a configfile of:

whitelist_from *@test@test.com *@blah@blah.com *@hui@hui.com

blacklist_from *@black@black.org *@black@white.org


which is more or less complete bogus.


1) for each entry of white/blacklist we need 1 whitelist_from/blacklist_from line, created with a for loop in tmpl, just as /etc/spamd/local.cf.tmpl does

2) the *@ is only necessary when an entry has only the domain

3) overlib description of these fields is also wrong:
we have only those 2 possibilities:
user@domain.org and
*@domain.org
(0003822)
baldy (reporter)
2010-02-21 17:46
edited on: 2010-02-21 17:47

 Peter,

Issue occurs also with whitelisting ip adresses.

Can this be added to this note or do we need to open another bugreport.

Regards,

Klaas-Jan
(0003826)
simon-endian (developer)
2010-02-22 17:12

 3) http://wiki.apache.org/spamassassin/ManualWhitelist [^] states 3 possibilities: user@domain.org, *@domain.org, *.domain.org
(0003827)
baldy (reporter)
2010-02-22 18:00
edited on: 2010-02-22 18:01

 Simon,

*@ fails with "*@newsletter.lidl.nl" at "Whitelist sender" is not valid!
*. also fails with "*.lidl.nl" at "Whitelist sender" is not valid!

Regards,

Klaas-Jan
(0003828)
simon-endian (developer)
2010-02-22 18:14

 Klaas-Jan,

This fails because to have the same syntax on all fields,
it is required to use the syntax from the help text examples.
in your case you would need to use newsletter.lidl.nl, .lidl.nl
and example@newsletter.lidl.nl
The restartscript then adds *@ or * to the values (only needed
for spamassassin, because it uses another syntax then for
example postfix).

The bug causing a not working configuration is located in the
restartsmtpscan.py script and the local.cf.tmpl of spamassassin.

Quickfix: spamassassin_manual_lists.patch (attached files)

Please try the quickfix and report back if it solves your problem.

Regards,
Simon
(0003829)
simon-endian (developer)
2010-02-22 18:16

 Klaas-Jan,

manual whitelist and blacklist of spamassassin does not support IPs.

3 fields are supporting IPs: Whitelist client and Blacklist client and whitelist client with greylisting. Which field (label) is not working for you?

Regards Simon
(0003831)
baldy (reporter)
2010-02-22 19:22

 Simon,

I have added a single ipaddress to whitelist client.

Just the ip, no /subnet.

Regards,

Klaas-Jan
(0003834)
baldy (reporter)
2010-02-22 22:05

 Simon,

How do I apply the patch ?

Regards,

Klaas-Jan
(0003835)
simon-endian (developer)
2010-02-23 09:20

 Klass-Jan,

I uploaded the files. you now need to replace /usr/local/bin/restartsmtpscan.py with restartsmtpscan.py and /etc/spamassassin/local.cf.tmpl with local.cf.tmpl .

Can you post the the output of:
cat /etc/postfix/client_rules


Regards,
Simon
(0003836)
baldy (reporter)
2010-02-23 09:34

 Simon,

Replaced both files.

Output from client_rules

root@mail:~ # cat /etc/postfix/client_rules
213.189.20.67 OK
195.238.6.0/24 REJECT
194.78.205.0/24 REJECT
212.53.0.0/20 REJECT
root@mail:~ #

Which file stores the emailaddresses from white/blacklist ?
/etc/spamassassin/Local.cf does not contain any.

Regards,

Klaas-Jan
(0003837)
simon-endian (developer)
2010-02-23 09:59

 output from client_rules should be correct. what is the wrong behavior? is the ip:213.189.20.67 not whitelisted? can you test with 213.189.20.67/32 instead?

did you define the white- / blacklist in the Spam (Black- & Whitelist) action?
/etc/spamassassin/local.cf holds the white- / blacklist defined at Spam (Black- & Whitelist)
Accepted mail (Black- & Whitelists) are stored at:
/etc/postfix/*_rules
* = sender, recipient, client

What behavior do you want:
- mails being white- / blacklisted from being marked as spam? -> define them in the spam section
- mails being white- / blacklisted to not get / get rejected regardless if they are spam or not -> define them in the accepted mail section

Regards,
Simon
(0003841)
baldy (reporter)
2010-02-23 15:48

 Simon,

All mailadresses are defined in the accepted mailsection.

I can not define them in the spam section as these are not saved.

Regards,

Klaas-Jan
(0003846)
baldy (reporter)
2010-02-24 13:33

 Simon,

I have tested with the /32 behind the ip address and various other possibilities, but still no go.

I have tested :

213.189.20.67/32
213.189.20.67
213.189.0.0/19
213.189.20.0/24
213.189.16.0/20

But mail still ends up in spam quarantine.

Headers from quarantined mail :

Received: from mail.baldy.nl (192.168.200.1) by remote.baldy.nl
 (192.168.200.4) with Microsoft SMTP Server (TLS) id 8.1.393.1; Wed, 24 Feb
 2010 14:29:31 +0100
Received: from localhost (localhost.localhost [127.0.0.1]) by mail.baldy.nl
 (Postfix) with ESMTP id 53999C5A83 for <spam@nospam_baldy.nl>; Wed, 24 Feb 2010
 14:29:31 +0100 (CET)
X-Envelope-From: <apache@nospam_mangrove.nl>
X-Envelope-To: <info@nospam_henkvangent.nl>
X-Envelope-To-Blocked: <info@nospam_henkvangent.nl>
X-Quarantine-ID: <8SSfpP1gVFD1>
X-Spam-Flag: YES
X-Spam-Score: 4.698
X-Spam-Level: ****
X-Spam-Status: Yes, score=4.698 tag=4 tag2=4 kill=4
    tests=[FORGED_HOTMAIL_RCVD2=1.117, HTML_MESSAGE=0.001,
    HTML_MIME_NO_HTML_TAG=1.052, MIME_HEADER_CTYPE_ONLY=0.856,
    MIME_HTML_ONLY=1.672] autolearn=no
X-Spam-Report: * 1.1 FORGED_HOTMAIL_RCVD2 hotmail.com 'From' address, but no 'Received:'
 * 0.0 HTML_MESSAGE BODY: HTML included in message
 * 1.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 * 0.9 MIME_HEADER_CTYPE_ONLY 'Content-Type' found without required MIME
 * headers
 * 1.1 HTML_MIME_NO_HTML_TAG HTML-only message, but there is no HTML tag
Received: from mail.baldy.nl ([127.0.0.1]) by localhost (mail.baldy.nl
 [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8SSfpP1gVFD1 for
 <info@nospam_henkvangent.nl>; Wed, 24 Feb 2010 14:29:27 +0100 (CET)
Received: from sh-1.mangrove.nl (berglind.mangrove.nl [213.189.20.67]) by
 mail.baldy.nl (Postfix) with ESMTP id 0DDA1C5A82 for <info@nospam_henkvangent.nl>;
 Wed, 24 Feb 2010 14:29:24 +0100 (CET)
Received: by sh-1.mangrove.nl (Postfix, from userid 48) id CBBD3400B0; Wed, 24
 Feb 2010 14:29:23 +0100 (CET)
To: info@nospam_henkvangent.nl
Subject: contact formulier www.henkvangent.nl
From: van der Borden <pijpfitter@nospam_hotmail.com>
X-Sender: <henkvangent.nl>
X-Mailer: henkvangent.nl
Content-Type: text/html; charset="iso-8859-1"
Message-ID: <20100224132923.CBBD3400B0@sh-1.mangrove.nl>
Date: Wed, 24 Feb 2010 14:29:23 +0100
MIME-Version: 1.0
Return-Path: <>
X-MS-Exchange-Organization-Antispam-Report: IPOnAllowList
X-MS-Exchange-Organization-SCL: -1
(0004306)
baldy (reporter)
2010-05-30 10:01

 Issue is fixed in 2.4

- Issue History
Date Modified Username Field Change
2010-02-15 12:24 baldy New Issue
2010-02-15 15:52 peter-endian Note Added: 0003794
2010-02-15 15:54 peter-endian Note Added: 0003795
2010-02-15 15:54 peter-endian Status new => feedback
2010-02-15 16:16 baldy Note Added: 0003797
2010-02-16 13:40 baldy Note Added: 0003805
2010-02-16 14:33 baldy Note Added: 0003807
2010-02-17 14:12 peter-endian Status feedback => confirmed
2010-02-17 14:25 peter-endian Note Added: 0003810
2010-02-17 14:25 peter-endian Status confirmed => new
2010-02-17 14:25 peter-endian Assigned To => simon-endian
2010-02-17 14:25 peter-endian Status new => confirmed
2010-02-17 14:25 peter-endian Target Version => 2.3.1
2010-02-21 17:46 baldy Note Added: 0003822
2010-02-21 17:47 baldy Note Edited: 0003822
2010-02-22 17:12 simon-endian Note Added: 0003826
2010-02-22 18:00 baldy Note Added: 0003827
2010-02-22 18:01 baldy Note Edited: 0003827
2010-02-22 18:14 simon-endian Note Added: 0003828
2010-02-22 18:15 simon-endian File Added: spamassassin_manual_lists.patch
2010-02-22 18:16 simon-endian Note Added: 0003829
2010-02-22 19:22 baldy Note Added: 0003831
2010-02-22 22:05 baldy Note Added: 0003834
2010-02-23 09:16 simon-endian File Added: restartsmtpscan.py
2010-02-23 09:17 simon-endian File Added: local.cf.tmpl
2010-02-23 09:20 simon-endian Note Added: 0003835
2010-02-23 09:21 simon-endian Status confirmed => feedback
2010-02-23 09:34 baldy Note Added: 0003836
2010-02-23 09:59 simon-endian Note Added: 0003837
2010-02-23 15:48 baldy Note Added: 0003841
2010-02-24 13:33 baldy Note Added: 0003846
2010-03-03 15:18 ra-endian Status feedback => assigned
2010-03-03 15:49 ra-endian Priority normal => high
2010-03-05 11:57 peter-endian Relationship added has duplicate 0002655
2010-05-30 10:01 baldy Note Added: 0004306
2010-05-30 10:02 baldy Status assigned => resolved
2010-05-30 10:02 baldy Resolution open => fixed
2010-11-22 12:08 peter-endian Fixed in Version => 2.4.1
2010-11-22 12:08 peter-endian Status resolved => closed

Copyright © 2005-2008 Endian, SRL. All rights reserved.

Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker