Implementação de filtro de borda com MailScanner
Introdução
Configuração do Mailscanner
- Mailscanner
- Spam Assassin
- Clam AV
- Postfix
Ambiente
Em produção no cliente CIMCOP
Distributor ID: CentOS Description: CentOS release 5.11 (Final) Release: 5.11 Codename: Final
Em testes no ambiente Ouro Minas
Distributor ID: OracleServer Description: Oracle Linux Server release 6.6 Release: 6.6 Codename: n/a
Repositórios YUM configurados
base CentOS-5 - Base epel Extra Packages for Enterprise Linux 5 - i386 extras CentOS-5 - Extras rpmforge RHEL 5 - RPMforge.net - dag updates CentOS-5 - Updates
Pacotes para instalação
Os scripts para instalação que são mencionados aqui ficam no servidor /opt/cit/work/MailGateway
Os pacotes devem ser baixados do site do fabricante dos softwares
# cd /opt/cit/work/MailGateway/
Instalando o MailScanner
Com a versão MailScanner-4.85.2-3 a instalação para variantes do RedHat/CentOS foi otimizada para ser via yum
# cd 10-MailScanner/src/MailScanner-4.85.2-3 # service MailScanner stop # bash install.sh
Atualizando o MailScanner.conf
MailScanner 4.85.2-3 - A atualização é feita automaticamente
# mv /etc/MailScanner /etc/MailScanner.BKP
Após a atualização com o RPM
# cd /etc/MailScanner # upgrade_MailScanner_conf ../MailScanner-BKP/MailScanner.conf MailScanner.conf > MailScanner.new
# mv -f MailScanner.conf MailScanner.ORI # mv -f MailScanner.new MailScanner.conf
Configurando o postfix
Foram comentadas algumas linhas do script, como a que baixa o pacote do clam e SA, executado o script e realizadas algumas tarefas manualmente
service sendmail stop chkconfig sendmail off yum install postfix cp -f main.cf main.cf-$(date +%j-%T) cp -f transport transport-$(date +%j-%T) cp -f header_checks header_checks-$(date +%j-%T) cd /etc/postfix
- /etc/postfix/main.cf
myhostname = mail2.cimcop.com.br
mydomain = cimcop.com.br
mydestination = $myhostname, localhost.$mydomain, localhost
local_transport = local
mynetworks = 127.0.0.0/8, 10.25.10.0/24
relay_domains = hash:/etc/postfix/relay_domains
header_checks = regexp:/etc/postfix/header_checks
# Para ativas as restricoes de smtpd_client
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = cimcop.com.br
broken_sasl_auth_clients = yes
### HELO RESTRICTIONS ###
smtpd_helo_restrictions = permit_sasl_authenticated,
permit_mynetworks,
check_helo_access hash:/etc/postfix/helo_checks,
reject_non_fqdn_hostname,
reject_invalid_hostname,
permit
### SENDER RESTRICTIONS ###
smtpd_sender_restrictions = permit_mynetworks,
reject_non_fqdn_sender,
reject_unknown_sender_domain,
permit
### RECIPIENT RESTRICTIONS ###
smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_unauth_pipelining,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
permit_mynetworks,
reject_unauth_destination,
reject_unverified_recipient,
permit
#check_policy_service inet:127.0.0.1:10031,
transport_maps = hash:/etc/postfix/transport
message_size_limit = 20480000
mailbox_size_limit = 20480000
- vi /etc/postfix/master.cf
smtp inet n - n - - smtpd submission inet n - n - - smtpd pickup fifo n - n 60 1 pickup cleanup unix n - n - 0 cleanup qmgr fifo n - n 300 1 qmgr tlsmgr unix - - n 1000? 1 tlsmgr rewrite unix - - n - - trivial-rewrite bounce unix - - n - 0 bounce defer unix - - n - 0 bounce trace unix - - n - 0 bounce verify unix - - n - 1 verify flush unix n - n 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - n - - smtp relay unix - - n - - smtp
-o smtp_fallback_relay=
showq unix n - n - - showq error unix - - n - - error retry unix - - n - - error discard unix - - n - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - n - - lmtp anvil unix - - n - 1 anvil scache unix - - n - 1 scache
Configuração adicional, notada em erros de mensagem de log
postconf -e alias_maps=hash:/etc/aliases
- vi /etc/postfix/transport
cimcop.com.br smtp:[10.25.10.1]
- vi /etc/postfix/header_checks
/^Received:/ HOLD
- vi /etc/postfix/relay_domains
cimcop.com.br OK
- vi /etc/postfix/helo_checks
cimcop.com.br REJECT You are not in machine localhost REJECT You are not me
postmap header_checks postmap transport postmap relay_domains postmap helo_checks
yum groupinstall "Ferramentas de Desenvolvimento" chkconfig postfix off chkconfig MailScanner on chown -R postfix.postfix /var/spool/MailScanner/incoming chown -R postfix.postfix /var/spool/MailScanner/quarantine mkdir /var/spool/MailScanner/spamassassin chown postfix.postfix /var/spool/MailScanner/spamassassin yum install perl-Mail-SPF yum install perl-Mail-SPF-Query #rpm -e spamassassin yum install perl-Parse-RecDescent yum install perl-Inline perl-Inline-ASM perl-Inline-Awk perl-Inline-BC perl-Inline-Basic perl-Inline-CPP yum install perl-Inline-Files perl-Inline-Filters perl-Inline-Python perl-Inline-Struct perl-Inline-TT yum install zlib zlib-devel yum install gcc-c++
Pacotes baixados pelo cpan
# cpan cpan> force install Inline::Guile
Migrando o MailScanner.conf com as definições atuais
cd /etc/MailScanner cp MailScanner.conf MailScanner.conf-NEW scp root@10.25.10.9:/etc/MailScanner/MailScanner.conf MailScanner.conf-ATUAL upgrade_MailScanner_conf MailScanner.conf-ATUAL MailScanner.conf-NEW > MailScanner.new mv MailScanner.new MailScanner.conf
Razor-agents-sdk
O Razor agentes já vem instalado com a versão 4.85, basta liberar o firewall, veja a seguir
#bunzip2 razor-agents-sdk-2.07.tar.bz2 [ -d razor-agents-sdk-2.07 ] && rm -rf razor-agents-sdk-2.07 tar -xvf razor-agents-sdk-2.07.tar cd razor-agents-sdk-2.07 perl Makefile.PL make make test make install [ -d razor-agents-sdk-2.07 ] && rm -rf razor-agents-sdk-2.07
Razor-agents
Razor Agents vem instalado no pacote RPM com a versão 4.85 (perl-Razor-Agent-2.85-6.el6.x86_64), liberar a porta do firewall da seguinte forma (shorewall)
- /etc/shorewall/rules
ACCEPT fw net tcp 2703
#bunzip2 razor-agents-2.84.tar.bz2 [ -d razor-agents-2.84 ] && rm -rf razor-agents-2.84 tar -xvf razor-agents-2.84.tar cd razor-agents-2.84 perl Makefile.PL make make test make install [ -d razor-agents-2.84 ] && rm -rf razor-agents-2.84
dcc
O DCC já vem instalado com a versão 4.85
yum install spamass-milter-postfix (Isto reinstala o spamassassin do repositório)
cd src [ -d dcc-1.3.104 ] && rm -rf dcc-1.3.104 [ -d dcc-1.3.155 ] && rm -rf dcc-1.3.155
wget http://www.dcc-servers.net/dcc/source/dcc.tar.Z gunzip dcc.tar.Z tar -xvf dcc.tar cd dcc-1.3.155 ./configure --disable-dccm make install [ -d dcc-1.3.155 ] && rm -rf dcc-1.3.155
Integrando o MTA postfix com o MailScanner
- vi /etc/MailScanner/MailScanner.conf
Run As User = postfix Run As Group = postfix Incoming Queue Dir = /var/spool/postfix/hold Outgoing Queue Dir = /var/spool/postfix/incoming MTA = postfix
chown postfix.postfix /var/spool/postfix chown -R postfix.postfix /var/spool/MailScanner/incoming chown -R postfix.postfix /var/spool/MailScanner/incoming chown -R postfix.postfix /var/spool/MailScanner/quarantine chown -R postfix.postfix /var/spool/MailScanner/quarantine
MailWatch
Transformando os scripts em wiki
BASE=$(pwd) VER="1.2.0-beta-5"
cd ${BASE}
gunzip -dc mailwatch-${VER}.tar.gz | tar -xvf -
cd mailwatch-${VER}
Dependendo da versão escolher entre o mysql-server ou mariaadb-server
yum install gd php-gd mariadb-server php-mysql
- vi /etc/php.ini
short_open_tag On safe_mode Off register_globals Off magic_quotes_gpc Off magic_quotes_runtime Off session.auto_start 0
Setup da base de dados do MySQL/MariaDB
service mysqld start
chkconfig mysqld on
USER=mailwath
PASS=mailwath
cd ${BASE}/mailwatch-${VER}
mysql -uroot -pcancella < create.sql
mysql -uroot -pcancella <<EOF GRANT ALL ON mailscanner.* TO mailwatch@localhost IDENTIFIED BY 'mailwatch'; GRANT FILE ON *.* TO mailwatch@localhost IDENTIFIED BY 'mailwatch'; FLUSH PRIVILEGES; EOF
mysql -umailwatch -pmailwatch mailscanner<<EOF
INSERT INTO users VALUES ('admin',md5('asminasgerais'),'Administracao via WEB','A','0','0','0','0','NULL');
EOF
cd MailScanner_perl_scripts cat MailWatch.pm | sed -e "s/'root'/'mailwatch'/" -e "s/= /= 'mailwatch'/" > /usr/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm
cd ../..
mv mailwatch-${VER}/mailscanner /var/www/html/mailscanner
cd /var/www/html/mailscanner
chown root:apache images
chmod ug+rwx images
chown root:apache images/cache
chmod ug+rwx images/cache
[ ! -d temp ] && mkdir temp
chown apache:apache temp
cp conf.php.example conf.php
- vi conf.php
...
define('DB_TYPE', 'mysql');
define('DB_USER', 'mailwatch');
define('DB_PASS', 'mailwatch');
define('DB_HOST', 'localhost');
define('DB_NAME', 'mailscanner');
...
define('QUARANTINE_USE_FLAG', true);
cd /opt/cit/work/MailWatch
Ajustando o ambiente do cron do MailWatch
Ver o README de $MAILWATH/tools/Cron_Jobs
------------------------------------------- Database cleanup of maillog records ------------------------------------------- db_clean.php add to /etc/cron.daily/ You need to edit conf.php the RECORD_DAYS_TO_KEEP definition. You need to edit the db_clean.php to reflect the location of the functions.php file
-------------------------------------------- Quarantine Maintenance -------------------------------------------- Remove the clean.quarantine quarantine_maint.php add to /etc/cron.daily/ You need to edit conf.php the QUARANTINE_DAYS_TO_KEEP definition. You need to edit the quarantine_maint.php to reflect the location of the functions.php file
-------------------------------------------- Quarantine Report -------------------------------------------- quarantine_report.php add to /etc/cron.daily You will need to edit the quarantine_report.php to reflect the location of the functions.php file
-------------------------------------------- MailScanner Rule Editor -------------------------------------------- msre_reload.crond See the instructions in tools/MailScanner_rule_editor/INSTALL Add to /etc/cron.d when you have properly configured and installed msre_reload.sh
- vi /etc/MailScanner/MailScanner.conf
Always Looked Up Last = &MailWatchLogging Quarantine Whole Message = yes Quarantine User = root Quarantine Group = apache Quarantine Permissions = 0660/ Is Definitely Not Spam = &SQLWhitelist Is Definitely Spam = &SQLBlacklist
cp mailwatch-${VER}/MailScanner_perl_scripts/SQLBlackWhiteList.pm /usr/lib/MailScanner/MailScanner/CustomFunctions/SQLBlackWhiteList.pm
- vi /usr/lib/MailScanner/MailScanner/CustomFunctions/SQLBlackWhiteList.pm
my($db_name) = 'mailscanner'; my($db_host) = 'localhost'; my($db_user) = 'mailwatch'; my($db_pass) = 'mailwatch';
echo "bayes_path /etc/MailScanner/bayes/bayes" >> /etc/MailScanner/spam.assassin.prefs.conf echo "bayes_file_mode 0660" >> /etc/MailScanner/spam.assassin.prefs.conf
mkdir /etc/MailScanner/bayes chown root:apache /etc/MailScanner/bayes chmod g+rws /etc/MailScanner/bayes spamassassin -D -p /etc/MailScanner/spam.assassin.prefs.conf --lint
Autenticação com o saslauthd
Dependências
cyrus-sasl-plain-2.1.22-7.el5_8.1 cyrus-sasl-md5-2.1.22-7.el5_8.1 cyrus-sasl-lib-2.1.22-7.el5_8.1 cyrus-sasl-2.1.22-7.el5_8.1
- vi /etc/sysconfig/saslauthd
SOCKETDIR=/var/run/saslauthd MECH=rimap FLAGS=" -r -O 10.25.10.1"
Ativar o serviço
service saslauthd start chkconfig saslauthd on
Testando o serviço
# testsaslauthd -u danton@cimcop.com.br -p cancella 0: OK "Success."
- vi /etc/sasl2/smtpd.conf
pwcheck_method: saslauthd mech_list: plain login
Testando os serviços
Monitore o serviço e verifique possíveis erros no log
service MailScanner restart; tail -f /var/log/maillog
Acesso a interface do mailwatch
http://mail2.cimcop.com.br/mailscanner
Ajustes de configuração do MailScanner
Alguns ajustes pós implementação do 4.85
Spamassassin
EXPERIMENTAL - 05/02/2016 11:02
# vi /etc/mail/spamassassin/configserver.cf score URIBL_SBL 5.0 score URIBL_AB_SURBL 5.0 score URIBL_OB_SURBL 5.0 score URIBL_PH_SURBL 5.0 score URIBL_SC_SURBL 5.0 score URIBL_WS_SURBL 5.0 score URIBL_JP_SURBL 5.0
Inserir as regras Cancella em /etc/mail/spamassassin/local.cf
# Regras locais - CI header LOCAL_HEADER_PFIND ALL =~ /pfind/i header LOCAL_HEADER_BULK ALL =~ /Precedence:.*bulk/i header LOCAL_HEADER_UNSUBSCRIB ALL =~ /unsubscrib/i score LOCAL_HEADER_PFIND 6.0 score LOCAL_HEADER_BULK 6.0 score LOCAL_HEADER_UNSUBSCRIB 6.0 describe LOCAL_HEADER_PFIND Teste Cancella describe LOCAL_HEADER_BULK Teste Cancella describe LOCAL_HEADER_UNSUBSCRIB Teste Cancella
DCC/Razor
perldoc Mail::SpamAssassin::Plugin::DCC
DCC - out-bound UDP port 6277 DCC - out-bound TCP port 587 (for reporting spam) Razor - out-bound TCP port 2703
cpan -i Razor2::Client::Agent
spamassassin --lint
Teste
cdcc info
Razor2
Avaliar estes procediemntos (Zimbra) para a solucao com o MailScanner
# yum install razor # yum install perl-Razor-Agent Criar pastas e dar permissões para o Zimbra: # mkdir /opt/zimbra/amavisd/.razor; chown -Rf zimbra:zimbra /opt/zimbra/amavisd/.razor Criar a conta Razor (acessar como usuário Zimbra): $ razor-admin -home=/opt/zimbra/amavisd/.razor -create $ razor-admin -home=/opt/zimbra/amavisd/.razor -discover $ razor-admin -home=/opt/zimbra/amavisd/.razor -register Habilitar o plugin: # vim /opt/zimbra/conf/spamassassin/v310.pre Descomentar a linha: loadplugin Mail::SpamAssassin::Plugin::Razor2 Para checar se o Razor startou com o Zimbra, abra um terminal e reinicie o amavis: # zmamavisdctl restart Enquanto no outro terminal verifique o log do Zimbra: # tail -f /var/log/zimbra.log
Pyzor
- Configuração do firewall
ACCEPT net fw udp 24441 # Pyzor (Danton) ACCEPT net fw tcp 24441 # Pyzor (Danton) ACCEPT fw net udp 24441 # Pyzor (Danton) ACCEPT fw net tcp 24441 # Pyzor (Danton)
Executar os passos abaixo
pyzor --homedir /etc/mail/spamassassin discover echo "test" | spamassassin -D pyzor 2>&1 | less
Inserir em /etc/mail/spamassassin/local.cf
pyzor_options --homedir /etc/mail/spamassassin
Verificando a conectividade
# pyzor ping
O check não funcionou na última instalação
# pyzor check
MailScanner -D --lint
# chmod 777 /var/spool/MailScanner/incoming/
Não localizei uma solução para o erro abaixo o ID 89 é do postfix
Can't set GID 89 at /usr/sbin/MailScanner line 1542.
Verificar o tamanho/número de arquivos no Spamassassin-Temp
MailWatch
MailWatch não exibe informações do Bayes Database Information
# /usr/local/bin/sa-learn --force-expire bayes: synced databases from journal in 1 seconds: 2159 unique entries (4137 total entries)
Quarantine
Parâmetros para manter as mensagens de quarentena legíveis.
Quarantine Dir = /var/spool/MailScanner/quarantine Quarantine Whole Message = yes Quarantine Whole Messages As Queue Files = yes Required SpamAssassin Score = 3 # # What to do with spam # -------------------- # store/delete/forward.... Spam Actions = store High Scoring Spam Actions = store
Para recuperação de mensagem em quarentena
- /var/www/html/mailscanner/conf.php
define('QUARANTINE_MAIL_HOST', 'mail2.ourominas.com.br');
define('QUARANTINE_FROM_ADDR', 'release@ourominas.com.br');
define('QUARANTINE_USE_SENDMAIL', true);
define('QUARANTINE_SENDMAIL_PATH', '/usr/sbin/sendmail');
- /etc/MailScanner/rules/spam.whitelist.rules
From: release@ourominas.com.br yes
- Inserir o email release@ourominas.com.br na White List do MailWatch
Configuração da verificação de SPF
Contribuição do Sr.Marco Brandão
Pré requisitos
Perl 5.6 NetAddr-IP 4 Mail::SPF (not Mail-SPF-Query) version 2.006 or later Sys::Hostname::Long
Instalação dos pacotes necessários
yum install perl-Mail-SPF perl-NetAddr-IP perl-Sys-Hostname-Long wget https://launchpad.net/postfix-policyd-spf-perl/trunk/release2.010/+download/postfix-policyd-spf-perl-2.010.tar.gz tar -xzvf postfix-policyd-spf-perl-2.010.tar.gz cd postfix-policyd-spf-perl-2.010 cp postfix-policyd-spf-perl /usr/local/lib/policyd-spf-perl
Teste do Mail::SPF, tive problemas com o Oracle Linux, veja abaixo em Problemas e soluções
perldoc Mail::SPF
Ajustes na configuração do postfix
- /etc/postfix/main.cf
... policy unix - n n - 0 spawn user=nobody argv=/usr/local/lib/policyd-spf-perl ...
- /etc/postfix/master.cf
...
policy_time_limit = 3600
...
### RECIPIENT RESTRICTIONS ###
smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_unauth_pipelining,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
permit_mynetworks,
reject_unauth_destination,
reject_unverified_recipient,
check_policy_service unix:private/policy,
permit,
...
Reiniciando o serviço e verificando a funcionalidade
# service MailScanner restart
Serviços de log do linux
# tail -f /var/log/maillog | grep policy
Via MailScanner, abra o pacote no cabeçalho terá indicações de verificação SPF
Problemas e soluções
No Oracle Linux mesmo com o pacote perl-Mail-SPF instalado o utilitário não estava encontrando o pacote Mail::SPF, forcei a instalação com o cpan
# cpan -i -f Mail::SPF
Postfix Greylisting Policy Server
Implementação do postgrey para GreyList policy server, contribuição do Sr.Marco Brandão
Instalação dos pacotes e dependências
Nesta implementação, Oracle Linux, instalei os pacotes do postgrey direto do yum e suas dependências, identifiquei que é a mesma vertente utilizada pelo marco do site http://postgrey.schweikert.ch/ oferecido no rpmforge. Neste pacote já vem com o script de start/stop padrão LSB
# yum install postgrey
Tive problemas com o Oracle Linux em alguns dos pacotes abaixo, veja Problemas e soluções abaixo
Installing: postgrey noarch 1.34-1.el5.rf rpmforge Installing for dependencies: perl-BerkeleyDB x86_64 0.43-1.el5.rf rpmforge perl-IO-Multiplex noarch 1.13-1.el5.rf rpmforge perl-Net-Server noarch 0.99-1.el5.rf rpmforge perl-Parse-Syslog noarch 1.10-1.el5.rf rpmforge
Iniciando o serviço e verificando a funcionalidade
Veja a em Problemas e soluções os ajustes que tive que fazer no script de start/stop, tanks Marco Brandão. Veja em Problemas e soluções
# chkconfig --add postgrey # chkconfig postgrey on # service postgrey start
Configurando o postfix
- /etc/postfix/main.cf
...
smtpd_recipient_restrictions = permit_sasl_authenticated,
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unauth_pipelining,
reject_unknown_recipient_domain,
permit_mynetworks,
reject_unauth_destination,
check_recipient_access hash:/etc/postfix/verify_domains,
reject_unverified_recipient,
check_policy_service unix:private/policy,
check_policy_service unix:postgrey/socket,
permit
...
Reiniciando o MailScanner e verificando a funcionalidade
# service MailScanner restart
Verificando a mensagem sendo greylisted
# tail -f /var/log/maillog | grep Greylisted
Verificando as mensagens liberadas pelo postgrey
# tail -f /var/log/maillog | grep postgrey
Verificando as mensagens que estão em greylist e ainda não tiveram um retry
# cat /var/log/maillog | /usr/sbin/postgreyreport --delay=300 # cat /var/log/maillog | /usr/local/sbin/postgreyreport --delay=300 --greylist-text="Policy restrictions; Greylist in action; try later"
Verificando os top 20 endereços de origem que estão aguardando o retry
# cat /var/log/maillog | postgreyreport | awk '{print $1}' | sort | uniq -c | sort -nr | head -n20
# cat /var/log/maillog | postgreyreport --greylist-text="Policy restrictions; Greylist in action; try later" | awk '{print $1}' | sort | uniq -c | sort -nr | head -n20
Verificando os top 20 destinatários que estão aguardando um retry para receber mensagens
# cat /var/log/maillog | postgreyreport | awk '{print $4}' | sort | uniq -c | sort -nr | head -n20
# cat /var/log/maillog | postgreyreport --greylist-text="Policy restrictions; Greylist in action; try later" | awk '{print $4}' | sort | uniq -c | sort -nr | head -n20
Tratamento de whitelist
Sintaxe do daemon
--whitelist-clients=FILE default: /etc/postfix/postgrey_whitelist_clients
--whitelist-recipients=FILE default: /etc/postfix/postgrey_whitelist_recipients
--auto-whitelist-clients=N whitelist host after first successful delivery
N is the minimal count of mails before a client is
whitelisted (turned on by default with value 5)
specify N=0 to disable.
- vi /etc/postfix/postgrey_whitelist_recipients
postmaster@ abuse@ secretaria@
Problemas e soluções
Problema com o Oracle Linux
Tive problemas com o pacotes do perl Oracle Linux, tive que fazer a instalação direto do CPAN.
cpan -i Net::Server cpan -i IO::Multiplex cpan -i BerkeleyDB cpan -i Parse::Syslog
Configurando o postgrey para usar o socket unix
- /etc/init.d/postgrey
... OPTIONS="--unix=/var/spool/postfix/postgrey/socket --delay=60" ...
Reinicie o serviço e confira se esta sendo executado
# service postgrey restart; tail -f /var/log/maillog