HttpWebRequest, its request stream, and sending data in chunks

View Comments

• I'm wondering how small is "small" and how large is "large" to ASP.NET.

• Thank you for submitting this cool story - Trackback from DotNetShoutout

• You blog post has been tagged and indexed at following for availability to larger audience. Thanks!
  http://delicious.com/LiveFramework
  http://social.msdn.microsoft.com/Forums/en-US/liveframework/thread/828d9a48-239a-4af8-8239-35931e514d37#page:3

• Good to note that if you use GetRequestStream (instead of BeginGetRequestStream), then BeginGetResponse will be a synchronous operation. Took me a while to figure that out since the documentation wasn't completely clear.

• fifth Edition
  in Chapter 19, there is one note:
  Note: It is important to realize that a single address space consists of one executable module and several DLL modules. Some of these modules can link to a static version of the C/ C++ run-time library, some of these modules might link to a DLL version of the C/C++ run-time library, and some of these modules (if not written in C/C++) might not require the C/ C++ run-time library at all. Many developers make a common mistake because they forget that several C/C++ run-time libraries can be present in a single address space. Examine the following code:
  VOID EXEFunc() {
  PVOID pv = DLLFunc();
  // Access the storage pointed to by pv...
  // Assumes that pv is in EXE's C/C++ run-time heap
  free(pv);
  }
  PVOID DLLFunc() {
  // Allocate block from DLL's C/C++ run-time heap
  return(malloc(100));
  }
  So, what do you think? Does the preceding code work correctly? Is the block allocated by the DLL's function freed by the EXE's function? The answer is: maybe. The code shown does not give you enough information. If both the EXE and the DLL link to the DLL C/C++ run-time library, the code works just fine. However, if one or both of the modules link to the static C/C++ run-time library, the call to free fails. I have seen developers write code similar to this too many times, and it has burned them all.
  ===========================================================================
  At first one thing the starup() function do is to initialize a heap used by malloc and free and low-level IO manipulation use in CRT, so when there is two copies CRT liberary in one process adress and there will be two heaps for two groups of malloc and free, and that one block memory grabed by malloc can only be freed by the very free() function in the same copy.
  but what free() is?
  in fact free()'s source code is like this:
  void free(void *ptr)
  {
  struct mem_control_block *free;
  free = ptr - sizeof(struct mem_control_block);
  free->is_available = 1;
  return;
  }
  every malloc()function return pointer to a block,however there are some bytes ahead the block conserved for system information, and this information is stored in a struct mem_control_block, so indeed the pointer malloc()function returned points to the byte after the mem_control_block.
  struct mem_control_block {
  int is_available; //a flag that specify whether this block can be used by any other application
  int size; //the size of the block
  };
  finnally, it seems no error in these code:
  VOID EXEFunc() {
  PVOID pv = DLLFunc();
  // Access the storage pointed to by pv...
  // Assumes that pv is in EXE's C/C++ run-time heap
  free(pv);
  }
  PVOID DLLFunc() {
  // Allocate block from DLL's C/C++ run-time heap
  return(malloc(100));
  }
  if the system doesn't allow a free() to free a block in heap of another copy of CRT, then how the system is able to do it?
  Thanks for your help.
  I'm struggle reading
  Jack

• Good points about HttpWebRequest. Another way to avoid buffering in HWR is to set the property HttpWebRequest.AllowWriteStreamBuffering=false. This way, the request wont buffer the request payload, no matter what the size of the payload.

• Unable to get the non-buffered mode verified working. Would appreciate if anyone has a chance to review:
  http://stackoverflow.com/questions/6313851/httpwebrequest-upload-using-stream-always-buffering
  Thanks, Chris

• This is code for upload file via httprequest
  static Stream formDataStream = null;
  private void button3_Click(object sender, EventArgs e)
  {
  DialogResult result = this.openFileDialog1.ShowDialog();
  if (result == DialogResult.OK)
  {
  string fileUrl = openFileDialog1.FileName;
  HttpWebRequest myreq =
  (HttpWebRequest)WebRequest.Create("http://www.ssss.com/video.xml");
  string boundaryString = "----WebKitFormBoundaryrqinj7B0Pj91NMg9";
  // Set the http request header \
  myreq.Method = WebRequestMethods.Http.Post;
  myreq.ContentType = "multipart/form-data; boundary=" + boundaryString;
  myreq.KeepAlive = true;
  myreq.Credentials = new NetworkCredential("username", "passwd");
  formDataStream = getFormDataStream(boundaryString, fileUrl); //get the stream to send via http POST
  myreq.ContentLength = formDataStream.Length;
  // start the asynchronous operation
  myreq.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), myreq);
  }
  }
  private static void GetRequestStreamCallback(IAsyncResult asynchronousResult)
  {
  HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
  // End the operation
  Stream reqStream = request.EndGetRequestStream(asynchronousResult);
  formDataStream.Position = 0;
  byte[] buffer = new byte[1024];
  for (int len; (len = formDataStream.Read(buffer, 0, buffer.Length)) > 0; )
  {
  reqStream.Write(buffer, 0, len);
  }
  // Start the asynchronous operation to get the response
  request.BeginGetResponse(new AsyncCallback(GetResponseCallback), request);
  }
  private static void GetResponseCallback(IAsyncResult asynchronousResult)
  {
  HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
  // End the operation
  HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
  Stream streamResponse = response.GetResponseStream();
  }
  But i am getting the error
  Message=The underlying connection was closed: An unexpected error occurred on a receive. Message=An established connection was aborted by the software
  in your host machine
  on this line
  byte[] buffer = new byte[1024];
  for (int len; (len = formDataStream.Read(buffer, 0, buffer.Length)) > 0; )
  {
  reqStream.Write(buffer, 0, len);
  }
  I tried the same code with setting KeepAlive as False, but no change in response. What i want is need to upload a large file as small chunks into server via http post from a desktop application
  FormDataStream contains the whole data to send and while i am trying to write the content to httprequest i am getting this error,
  I follow the comments on 2nd point from thi s article

• I have an application and it seems we have no size limit to the files to be sent whant should we use for small payloads and what is ok to send via non-chunked channel

• Even though httpwebrequest is probably depreciated, this is some really interesting info that I was curious and confused about; thanks.