Those who work with the. NET framework and occasionally used the HttpWebRequest class, may have stumbled about the phenomenon that it seems to be quite slow in some cases. Especially if you use HttpWebRequest in combination with threading to get responses as quick as possible, the HttpWebRequest class quickly becomes a party pooper.
Initially you might assume that the problems rely on the Internet connection, but at least after a speed comparison with wget or curl, you should recognize that the “problem” must be at the. NET implementation.
No products found.
In the following article I will focus on three points that can help you, to speed up the HttpWebRequests class.1. Increase DefaultConnectionLimit
All instances of the HttpWebRequest class are using some properties of the ServicePointManager. One of these porperties of the ServicePointManager is the “DefaultConnectionLimit”. By default, this limit is set to 2, which means that there will be never more than 2 simultaneous connections.
Especially if you want to check multiple requests simultaneously (e.g. by threading) , this becomes an absolute brake. In this case, the DefaultConnectionLimit should be scaled up wisely and according to the capacity of the available network connection.
ServicePointManager.DefaultConnectionLimit = connectionLimit;
To find the optimal value, you should make multiple test runs with different values.
2. Make proxy settings correctly
Another bottle-neck may be the proxy server settings. If you search the internet for problems with HttpWebRequest, you will find frequently a problem description, which says, that only the first query takes a long time and all further queries go relatively quickly. The reason for that is, that the first query, after instantiation of the the HttpWebRequest, is looking for the system-wide proxy settings. This search progress can easily take 2-8 seconds. There are two solutions for that.
No proxy needed
If no proxy is needed, it can be explicitly set to null, so it will not look for the system-wide default proxy.
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://mywebsite.com");
req.Proxy = null;
It will need a proxy
If a proxy server is required and it’s settings are known (what should be the case, if you program in an environment where a proxy is needed) then this settings can be set manually. So the slow search progress for the proxy settings can also be bypassed.
string proxyUrl = "proxy.myproxy.com";
int proxyPort = 8080;
WebProxy myProxy = new WebProxy(proxyUrl, proxyPort);
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://mywebsite.com");
req.Proxy = myProxy;
3. Deactivate Expect100Continue
The last tip may also bring you a couple of seconds. More specifically – about 1/3 second per query. Depending on the number of queries this can be a good enhancement, too. In the default configuration a Continue(100) status is sent before every query. If the server is ready/willing to process our query, our client receives the response from the server and sends the actual request.
If we assume that we know the server and know that he will accept our query, we can disable the Continue(100) requests. In addition, there is still some server-software that does not get along with Continue(100) hits. More information about that can be found on this webpage.
ServicePointManager.Expect100Continue = false;
That’s it. Should you know other ways to accelerate HttpWebRequests, so just write me a comment.
wow this is really precious!
You have to know that http connections are limited to 2 by default. This might slow down all async work if you don’t have this in mind …
Exceptional Help RAFFAEL…Sir, pls elaborate me before getting the response how come i can read response as incremental webresponse message from webrequest by POST method in async HttpwebRequest?..now,I am trying to get response as,
using (var requestStream = await Task.Factory.FromAsync(request.BeginGetRequestStream, request.EndGetRequestStream, request))
{
await requestStream.WriteAsync(postDataBuffer, 0, postDataBuffer.Length);
}
HttpWebResponse response = (HttpWebResponse)await Task.Factory.FromAsync(request.BeginGetResponse, request.EndGetResponse, request);
Stream responseStream = response.GetResponseStream();
Rafael, on Nov 21/2016, the section “No proxy needed” save my day. Thank you very much, David Meritano from Buenos Aires, Argentina.
I’m pleased to read that this blog article was helpful for you. :-)
Its 2016 and this information helped me.
If I was not bald, I would have been if I had not read this article ;)
Thank you!
i just found this article. I hope it helps me. Thinking of web request and response, performance and speed must be the concern in case of parallel programming. Also doing job in big company give you extra responsibility about speed and performance.
Thank in advance.
Full regards.
Hi,
If i write below tag in windows config than will it be executed every time when below code is executed. So every time below code is executed i will be allowed to have max 1000000 no of parallel request/response.
HttpWebRequest POSTRequest = (HttpWebRequest)WebRequest.Create(“http://localhost:6665/AdAgent/ADImportUsers”);
Tag
Omg, I just found this article, it helped my stock app performance tremendously.
Thanks a lot!
Hi,
Many thanks for these explanations, I would like to add another case.
For me GetRequestStream() was taking 300ms evey time I called it, I just have to stop Web Anti-Virus Protection in Kapersky Anti-Virus 6.0.
I hope it helps somebody.
JP.