I have an app which is programmatically generating and sending out emails. The recipients list can get upwards of 1000. I had been looping through and sending out individual emails, but that was taking too long at about .5 sec each. The approach I'm looking into now is to remove customization in the message body and send out a single email with all addresses in the BCC. (Maybe other solutions are possible and I welcome them, but I'm mainly interested in delving into the complexities of this BCC solution.)
Is there a limit to the number of recipients allowed on a single email? Is this wholly dependent on my email client and/or SMTP server's configuration? Are there other limits outside the control of my domain? Furthermore, how is BCC handled? I assume that the BCC distribution needs to be broken down into separate mail messages at some point. Is the mail client (in my case javax.mail) responsible for this, or does the mail server do this?
I'm also interested in suggestions of how can I test my new email blaster program?
I don't think it will be a valid test by creating 1000 accounts at google or wherever (nor do I want to). I've heard there are some mail server optimizations geared toward multiple recipients at the same host. In my case, most will be distinct hosts.
Another way is to involve all the recipients to see if they recieved the email. I can do this, but I want to avoid spamming them, assuming I may need to test multiple times if things don't go right the first time.
Or do I just assume some limitation and send out batches of emails with some arbitrary number of recipients each, say 50 or 100?
BCC works inside your SMTP server; no recipients ever know other BCC'ed email addresses, so this is a limitation that depends entirely on your SMTP server.
You should check with your server administrator.
even more definitive, the RFC that covers SMTP (2821) makes no mention of recipient limitations outside of mail-server specific ones:
"If an SMTP server has an implementation limit on the number of RCPT
commands and this limit is exhausted, it MUST use a response code of
452 (but the client SHOULD also be prepared for a 552, as noted
above). If the server has a configured site-policy limitation on the
number of RCPT commands, it MAY instead use a 5XX response code.
This would be most appropriate if the policy limitation was intended
to apply if the total recipient count for a particular message body
were enforced even if that message body was sent in multiple mail
transactions."
http://www.ietf.org/rfc/rfc2821.txt
Thanks for your comments. As I understand it now, the outgoing SMTP server will be responsible for breaking up each of the messages. In construction of the new messages, the outgoing SMTP server will only send applicable RCPT TO commands for each BCC recipient. So in the case where all recipients are BCC, there will be only one RCPT TO command for each message.
That being the case, it seems like I really only need be concerned about our outgoing SMTP server configuration. No need to worry about the destination SMTP servers.
I got a suggestion that seems like a good way to test this. I could send my message to a number of recipients, each with a unique bogus child domain of our valid parent domain. When no MX record is found for the child, the parent will be used. The outgoing SMTP server will not be aware that the bogus domains don't exist, so this should avoid any SMTP optimatization for multiple recipients with common domains. We can probably also configure that these messages will all get routed to the same inbox.
Related
I'm currently trying to write a simple Spring email verification microservice. I'm using Apache SMTPClient API to connect to an SMTP server and verify the email address existence (connect -> login -> setSender -> setRecipient -> logout -> disconnect). When connecting to most big email providers (gmail, yahoo, etc.) there are no issues, but for others there's a bizarre 5-20s delay when connecting. For example, with protonmail I sometimes wait for what looks like a fixed 5s delay, sometimes I connect normally after ~150ms. With some other, smaller services it's a fixed delay every time I try to connect.
Since I'm doing my best to reduce the time required for such verification, this is a pretty massive issue, given it generally takes <2s when no such issues are encountered.
I was trying to find a reason for this delay, and I encountered 2 potential explanations:
tarpitting - while the delay it causes fits the issue, I am sending a singular connection request, while the mechanism is supposed to prevent spam/bulk emails,
being blacklisted - that also could be the cause, but in general that information is put into a (negative) reply to one of the SMTP commands I send.
I would appreciate any suggestions on what could be causing the delay and how to avoid it (if it's even possible).
I made a couple of SMTP connections to mail.protonmail.ch, and it is definitely doing some things that are unusual. I think they're implementing a variation of tarpitting.
For instance, sometimes after the connecting, mail.protonmail.ch immediately responds with an SMTP banner (that looks very much like a 220 response):
220-mailin008.protonmail.ch ESMTP Postfix
then, after a seemingly random delay, an actual 220 response:
220 mailin008.protonmail.ch ESMTP Postfix
Other times, after a seemingly random delay, it responds with a 220 response without sending a banner first:
220 mailin025.protonmail.ch ESMTP Postfix
Many client SMTP programs that are used to send spam do not implement SMTP properly. A shoddy program might 'get fooled' by the unusual way that mail.protonmail.ch responds, and send EHLO before it should. Then, mail.protonmail.ch can drop the connection, on the assumption that the client is not a 'real' SMTP MTA. A real MTA, however, would wait for the 'real' 220 response, before sending EHLO.
In this case I was able to solve the problem by simply using sockets - if all you need to do is verify the existnace of an email by sending RCPT TO command, this is the best way to do it. I personally could not find any Java APIs that both avoided the problem I had and allowed for easy sending of SMTP commands.
Well I am new to this and I don't know how to do it, so my senior fellows please help!!!!
There is a situation described below:
An HTTP client is sending a request (Request can be of any type, not concerned regarding the request type) that directly hits a Loadbalancer. The Loadbalancer then redirects the traffic, based on the load of the traffic, towards a "Gateway" system running in two V440 Server, GW logic is written in Java, that actually logically routs this request towards another two server nodes which actually process the request.
Now the scene is something like that: there are several parallel connections are established with this Gateway from several HTTP clients. One connection per client. It has been observed that, while making connections to this GW, in case of some clients the CPU utilization is going 98-99%.
Client is creating one connection with the GW on particular port. Opens a socket connection:
ServerSocket _ss = new ServerSocket(_port);
Socket s = _ss.accept();
and then GW waits for the input to come from the client.
Now my question is:
Why this kind of situation is happening, as it seems all fine
for rest of the clients and there connections.
Only few clients who are creating connections with the GW is making the situation?
Is there anyway we can track this client's IP so that we can understand if this
has been occurred by same clients every time?
Is there any resolution for this?
Since it is not happening for all the clients, we are certainly not going to find an immediate answer for this. However, this is what my limited research on the question yields
Firstly, Question 2
Configure your F5 to capture the client's IP. Since it is HTTP, there are multiple ways of tracking the requests. One is to
sniff the header X-FORWARDED-FOR which will give the client's IP
Address
Or try adding this rule in your logging engine
when CLIENT_ACCEPTED { log local0. "clientIP:[IP::client_addr] accessed" }
If you also need other data such as resources you can use one of the
other events such as HTTP_REQUEST:
when HTTP_REQUEST {
log local0. "clientIP:[IP::client_addr] accessed [HTTP::host][HTTP::uri]" }
Refer link for above here
Secondly, Question 1
For this you need to look at your available traffic statistic mechanisms. I read this, this and this. Enable the statistics, monitor them live, test, mock requests and analyze the output. I do not know of any other options other than this, right now.
Another option, if you can modify your Java program is to include some sort of performance logging mechanism for each request. But this means there is a lot of development and that I do not recommend at all.
Thirdly, Question 3
This is primarily opinion based. As far as I think if you figure out the problem, you can resolve it.
today i'm researching about the jmeter.
connect the gmail
login
send email
I passed 2 step and got error "java.net.UnknownHostException: mail.google.com" on step 3
Login :
Send Email :
I connected the link "mail.google.com" on the website.
Can somebody teach me what am i wrong ?
I would recommend using SMTP Sampler instead of simulating sending an email using web interface.
The relevant configuration would be:
Server: smtp.googlemail.com
Port: 587
Address from: your Gmail address
Address to: recipient address(es)
Check "Use Auth" and provide your full GMail username and password
Tick "Use StarTLS" radiobutton
Other fields are pretty much self-explanatory
References:
Set up Gmail with Outlook, Apple Mail, or other mail clients
Load Testing Your Email Server: How to Send and Receive E-mails with JMeter
I know this is not an answer you will like, but you should not be pointing any performance testing tool at a server you do not own, control or manage (or have permission from the people who do). In addition, Google's own end user agreement prohibits this behavior, supporting automation only with their published API set. License-wise and ethically, this is a problematic set of behavior for performance testing.
If you wish to experiment, then consider setting up a LAMP stack box inside of your own domain/control with an SMTP relay, pointing to mailbox destinations on that server. If you must test routing then setup two email servers with destinations on the opposite host (easy with two virtual machines).
Note, SMTP supports direct addressing, such as [name]#[host|IP address] so you can control directly any email routing to outside mailboxes accidentally and SPAMming individuals or organizations. This also avoids the problem of clogging up an SMTP relay with thousands of messages undeliverable due to lack of a valid address - this last condition will slow all email delivery for an organization and if not addressed can grind delivery to a halt before the first undeliverable messages start failing out after the delivery timeout is reached which is typically around 72 hours.
So the problem is I have fifteen clients which need to be able to communicate between each other. My question is how should this be done? Clearly one way is to simply make the clients also servers, but that means 120 unique connections necessary to fully connect the fifteen clients. I'd rather not do this as it seems messy.
Current solution:
Each new connection has the server spin off a separate thread for listening to it. Each client has a separate thread monitoring the channel for incoming information.
Server acts as a message router: Process 1 needs to send a message to Process 2 and sends a message to the server indicating intended recipient, sender, and message.
Upon receiving the message the server passes message to Process 2. The listening thread detects it and passes it to the process.
So on for each message between the clients.
This seems clunky. Is there a better methodology/package to use for this?
A UDP multicast system would work for this but will get complicated for you to do yourself (since you have to worry about synchronization and fault detection/correction yourself as well as nodes droping in and out of the group).
There are various middleware solutions including distributed caches that already address this problem pretty well. Look at Infinispan. If that's too high level and you just want a lower level solution, try JGroups. I only list those because I know they are quick and usable, but there are many others out there.
Scenario 1 :
Setup a JMS Queue in your server
Java code to send Messages to Producer
Create a JMS Producer, which when invoked, should receive the email data (subject, body, to , cc etc) and post it to the Queue setup in step 1
Create a JMS Consumer, which subscribes to the Queue created in Step 1, and its onMessage should call the JavaMail API to send the email.
Scenario 2 :
Directly call the JavaMail API to send the email.
I know about how to use and what JMS and Java Mail are doing.Thing is why we have to go from Scenario 2 to Scenario 1 for sending mails.Initially we did Scenario 2.Now we are using Scenario 1.From Different parts of the Big Application are sending mails so we use JMS Queue ,there will be Consumer of Queue from there sending mails.Please help me to understand.
You would use this mechanism in a large application for 2 reasons:
1) You don't want your clients to have to wait for the mail to be sent.
2) You don't want to lose mails if you lose connectivity to your mail server for any reason.
You would do this if you don't have a relyable MTA near your local machine but need to be sure your mail will be send. For example if there is a network outage but you rely on Java Mail to send your mail without additional logic, your mail will not be send at all.
Using JMS you can reschedule the mail for transfer as soon as the real MTA will become available again.
Besides:
the conversation with the mail provider (SMTP und POP3) is
asynchronous and close to the JMS/MDB api. So why should i use a
different API than JMS ?
You can keep the mail handling in one transaction, together with some database changes other activities. I remember too many Spring .. sic' projects, where the customer demmands for a atomic operation, that included a state change in a db ;-)
Image, the messages you send become more compulsory and you have to connect to a X400 service. Simply think of the slight code change (and the change of the RA) and you will discover to met the right architectual descision.