Can send emails locally but not to anyone else

I've set up postfix + dovecot on sheffieldaustralia.com and can send/receive emails locally, but when I send one to anyone else they don't arrive. And when I email ben@sheffieldaustralia.com I get a bounce with "No Such User Here".

Any ideas on what to try? I've reviewed all the setup steps on Linode's guides several times, and think I'm at the point of diminishing returns with that approach.

Sample stuff from syslog:

> Feb 14 03:45:47 coagulopath dovecot: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/mysql.conf.d' directive as maximum includerecursion level was reached in file /etc/mysql/mysql.conf.d/mysqld.cnf at line 21!

Feb 14 03:45:47 coagulopath postfix/submission/smtpd[30751]: 7A17E2153E: client=unknown[203.217.17.183], saslmethod=PLAIN, saslusername=ben@sheffieldaustralia.com

Feb 14 03:45:47 coagulopath postfix/cleanup[30757]: 7A17E2153E: message-id=<e5259665-b869-b3c3-906d-5f76186eec9b@sheffieldaustralia.com>

Feb 14 03:45:47 coagulopath postfix/qmgr[18202]: 7A17E2153E: from=<ben@sheffieldaustralia.com>, size=624, nrcpt=1 (queue active)

Feb 14 03:45:47 coagulopath dovecot: lmtp(30761): Connect from local

Feb 14 03:45:47 coagulopath dovecot: auth-worker(30755): Warning: mysql: Query failed, retrying: Unknown column 'home' in 'field list'

Feb 14 03:45:47 coagulopath dovecot: auth-worker(30755): Error: sql(mail@ben-ts.net): User query failed: Unknown column 'home' in 'field list' (using built-in default user_query: SELECT home, uid, gid FROM users WHERE username = '%n' AND domain = '%d')

Feb 14 03:45:47 coagulopath dovecot: lmtp(mail@ben-ts.net): Error: user mail@ben-ts.net: Auth USER lookup failed

Feb 14 03:45:48 coagulopath postfix/lmtp[30760]: 7A17E2153E: to=<mail@ben-ts.net>, relay=coagulopath.sheffieldaustralia.com[private/dovecot-lmtp], delay=0.74, delays=0.72/0.01/0.01/0.01, dsn=4.3.0, status=deferred (host coagulopath.sheffieldaustralia.com[private/dovecot-lmtp] said: 451 4.3.0 <mail@ben-ts.net> Internal error occurred. Refer to server log for more information. (in reply to RCPT TO command))

Feb 14 03:45:48 coagulopath dovecot: lmtp(30761): Disconnect from local: Successful quit

Feb 14 03:45:48 coagulopath postfix/submission/smtpd[30751]: disconnect from unknown[203.217.17.183] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8

Feb 14 03:45:49 coagulopath dovecot: imap(ben@sheffieldaustralia.com): Connection closed in=1008 out=2481

Feb 14 03:46:07 coagulopath systemd[1]: Started Session 1447 of user root.

Feb 14 03:46:09 coagulopath dovecot: auth-worker(30755): Warning: mysql: Query failed, retrying: Unknown column 'home' in 'field list'

Feb 14 03:46:09 coagulopath dovecot: auth-worker(30755): Error: sql(ben@sheffieldaustralia.com,203.217.17.183): User query failed: Unknown column 'home' in 'field list' (using built-in default user_query: SELECT home, uid, gid FROM users WHERE username = '%n' AND domain = '%d')

Feb 14 03:46:09 coagulopath dovecot: imap-login: Login: user=<ben@sheffieldaustralia.com>, method=PLAIN, rip=203.217.17.183, lip=198.58.115.175, mpid=30795, TLS, session=

Originally, dovecot was complaining that it couldn't find "mailserver.users" (as I'm using virtual hosts, it should be mailserver.virtual-users, right?). I created a "users" table to make it happy and now it's asking for a "home" column. Clearly something is wrong with my setup.

Not sure where my "server log" is. There's nothing with a name similar to that in /var/logs

Could DNS issues be to blame? Here's my output when I dig sheffieldaustralia.com mx

> ; <<>> DiG 9.10.3-P4-Ubuntu <<>> sheffieldaustralia.com mx

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20257

;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 5, ADDITIONAL: 5

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 4096

;; QUESTION SECTION:

;sheffieldaustralia.com. IN MX

;; ANSWER SECTION:

sheffieldaustralia.com. 300 IN MX 20 mail.sheffieldaustralia.com.

sheffieldaustralia.com. 300 IN MX 10 sheffieldaustralia.com.

;; AUTHORITY SECTION:

sheffieldaustralia.com. 300 IN NS ns4.linode.com.

sheffieldaustralia.com. 300 IN NS ns2.linode.com.

sheffieldaustralia.com. 300 IN NS ns3.linode.com.

sheffieldaustralia.com. 300 IN NS ns1.linode.com.

sheffieldaustralia.com. 300 IN NS ns5.linode.com.

;; ADDITIONAL SECTION:

sheffieldaustralia.com. 300 IN A 198.58.115.175

sheffieldaustralia.com. 300 IN AAAA 2600:3c00::f03c:91ff:fe29:edc7

mail.sheffieldaustralia.com. 300 IN A 198.58.115.175

mail.sheffieldaustralia.com. 300 IN AAAA 2600:3c00::f03c:91ff:fe29:edc7

;; Query time: 91 msec

;; SERVER: 96.126.124.5#53(96.126.124.5)

;; WHEN: Wed Feb 14 15:02:07 AEDT 2018

;; MSG SIZE rcvd: 273p

Any ideas on what I should try first?

Thanks

  • Ben

10 Replies

Dovecot is trying to do something it's not supposed to be doing with its configuration. Examine its startup scripts and configuration carefully (including /etc/default). Maybe you have a conf.d symlink somewhere that points to mysql configs?

I partially fixed the problem, somehow.

  • I changed my FQDN in my hosts file (not sure how I missed this, it was still set to ubuntu.members.linode.com)

  • I deleted domain records at my old shared hosting platform

  • I added an MX record for my FQDN

One of those three things (which one? Answer and win a prize!) enabled me to receive inbound mail. I still can't send outbound mail, that's the next thing to look at.

So I forgot to comment stuff in my master.cf. I have fixed this and now I receive an error message when sending remotely.

> 5.1.1 : Recipient address rejected: User unknown in virtual mailbox table.

Please check the message recipient "" and try again

What could this mean?

It's like postfix/dovecot only knows to look inside its own local mailbox table.

````
Feb 14 03:45:47 coagulopath dovecot: auth-worker(30755): Warning: mysql: Query failed, retrying: Unknown column 'home' in 'field list'
Feb 14 03:45:47 coagulopath dovecot: auth-worker(30755): Error: sql(mail@ben-ts.net): User query failed: Unknown column 'home' in 'field list'

Your database schema is severely misconfigured. On top of that, dovecot seems to not be reading the configuration correctly:

Feb 14 03:45:47 coagulopath dovecot: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/mysql.conf.d' directive as maximum includerecursion level was reached in file /etc/mysql/mysql.conf.d/mysqld.cnf at line 21!
````

At a guess, postfix fails to invoke the dovecot-supplied user lookup and falls back to the default scheme. On top of that you probably have misconfigured postfix installation as well. Please post complete main.cf, master.cf and dovecot configuration sans sensitive passwords.

Thanks for the help.

> Your database schema is severely misconfigured. On top of that, dovecot seems to not be reading the configuration correctly:

I think my schema is OK. I've dropped the mailserver.users table (as I've decided it was a red herring), now there's just mailserver.virtual-users, mailserver.virtual-domains, and mailserver.virtual-aliases.

Dovecot definitely isn't reading the right table. postfix/mysql-virtual-email2email.cf is configured to point to here:

> query = SELECT email FROM virtual_users WHERE email='%s'

dovecot/dovecot-sql.conf.ext is configured to point to here:

> passwordquery = SELECT email as user, password FROM virtualusers WHERE email='%u';

Despite this, dovecot still insists on looking in mailserver.users, instead of mailserver.virtual-users. Is there some third file I have to change?

> At a guess, postfix fails to invoke the dovecot-supplied user lookup and falls back to the default scheme. On top of that you probably have misconfigured postfix installation as well. Please post complete main.cf, master.cf and dovecot configuration sans sensitive passwords.

main.cf

https://pastebin.com/ypraRmvb

master.cf

https://pastebin.com/gpLJ8y8j

dovecot.conf

https://pastebin.com/ZcctTmBe

auth-sql.conf

https://pastebin.com/2pTQzUb3

10-auth.conf

https://pastebin.com/QwXy40xn

dovecot.sql.conf.ext

https://pastebin.com/qRsrN5rK

Anything else I should post?

Hold the phone, I've had an idea.

The domain of the receiver email address I've been testing is also registered inside my virtual hosts. (I'm in the process of moving over an existing shared hosting account).

I suspect that some version of this is happening:

I send out an email to someone@domain.com, postfix notes that "domain.com" is also present inside localhost, and tries to route it internally. This fails, because someone@domain.com isn't present inside my mysql tables. It's like if I had a grandma in North Korea, but I also happen to live in a suburb of Australia called "North Korea". I've sent her a letter addressed "Grandma, North Korea", and the local post office is now attempting to deliver it to my local suburb, rather than the country.

To test my theory, I emailed a totally different email address that is unrelated to any domain configured on my Linode, and it apparently went through (still have to talk to this person to see that they actually received the email).

So maybe there was never a problem at all. I'll move the domain across, and that will be that. Unless I'm overlooking something else. Still no idea about the user/virtual-user table issue, but if my email works…

Personally, I avoid using an SQL database with postfix/dovecot. My suggestion is to link postfix to dovecot and use plain file storage. This has several benefits, the setup is much much simpler, there is little chance of getting problems in the future and you don't depend on running an SQL database. This is important, because if your database dies, then your emails go down with it.

There are benefits to using an SQL database, but usually these are special cases and the sysops know what they are doing.

````
Feb 14 03:45:47 coagulopath dovecot: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/mysql.conf.d' directive as maximum includerecursion level was reached in file /etc/mysql/mysql.conf.d/mysqld.cnf at line 21!

It looks like you're trying to include mysql configuration files into dovecot. At a guess, it fails, and so you get default settings and the query against the schema you do not implement (note the bolded part):

Feb 14 03:45:47 coagulopath dovecot: auth-worker(30755): Error: sql(<email email="mail@ben-ts.net">mail@ben-ts.net</email>): User query failed: Unknown column 'home' in 'field list' (****using built-in default**** user_query: SELECT home, uid, gid FROM users WHERE username = '%n' AND domain = '%d')

I can not find this in the settings you pasted, so it's either included from a file that you haven't pasted or there's something horribly wrong with how dovecot services are started. Find the culprit by:

grep -R mysql.conf.d /etc/dovecot

You should also revert postfix mydestination for good order:

mydestination = $myhostname, localhost
````

Also clean up your DNS. Remove the extra MX and only leave the one pointing to mail.xxx.com

@IfThenElse:

Personally, I avoid using an SQL database with postfix/dovecot. My suggestion is to link postfix to dovecot and use plain file storage. This has several benefits, the setup is much much simpler, there is little chance of getting problems in the future and you don't depend on running an SQL database. This is important, because if your database dies, then your emails go down with it.

There are benefits to using an SQL database, but usually these are special cases and the sysops know what they are doing.

SQL is used here as an authentication backend only. Storing mail contents in the database would be just silly.

That being said my standard suggestion is to stop mucking around with hand-crafted configuration and use iredmail.

> grep -R mysql.conf.d /etc/dovecot

No results.

> Also clean up your DNS. Remove the extra MX and only leave the one pointing to mail.xxx.com

Done. Do I need my mailserver at mail. or can it be at my root domain?

Also, here's what I get when I type service dovecot status.

> Feb 16 10:05:44 coagulopath dovecot[7180]: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/conf.d/' directive as maximum includerecursion level was reached in file /etc/mysql/conf.d/mysql.cnf at line 23!

Feb 16 10:05:44 coagulopath dovecot[7180]: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/conf.d/' directive as maximum includerecursion level was reached in file /etc/mysql/mysql.conf.d/mysqld.cnf at line 20!

Feb 16 10:05:44 coagulopath dovecot[7180]: auth-worker: Error: [Warning] skipping '!includedir /etc/mysql/mysql.conf.d' directive as maximum includerecursion level was reached in file /etc/mysql/mysql.conf.d/mysqld.cnf at line 21!

Feb 16 10:05:44 coagulopath dovecot[7180]: auth-worker(25633): Warning: mysql: Query failed, retrying: Table 'mailserver.users' doesn't exist

Feb 16 10:05:44 coagulopath dovecot[7180]: auth-worker(25633): Error: sql(mail@ben-ts.net): User query failed: Table 'mailserver.users' doesn't exist (using built-in default user_query: SELECT home, uid, gid FROM u

Feb 16 10:05:44 coagulopath dovecot[7180]: lmtp(mail@ben-ts.net): Error: user mail@ben-ts.net: Auth USER lookup failed

Feb 16 10:05:44 coagulopath dovecot[7180]: lmtp(25631): Disconnect from local: Successful quit

Feb 16 10:08:34 coagulopath dovecot[7180]: imap(admin@coagulopath.com): Disconnected for inactivity in=100 out=557

Feb 16 10:09:25 coagulopath dovecot[7180]: imap(ben@sheffieldaustralia.com): Logged out in=106 out=564

Feb 16 10:09:25 coagulopath dovecot[7180]: imap(mail@ben-ts.net): Connection closed in=100 out=521

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct