Stream Entity Framework results from WebAPI

Generally we try and keep our web services slim, passing back and forth as little data as possible.

However, there are times where we need to send back large amounts of data directly from a database.

If you try and pull all the data into memory before sending it, you can quickly run into trouble.  DB server load, web server load, and memory can all be exhausted very easily.

The answer is to stream the data back to the client.

If you are using WebAPI and Entity Framework, here is a small example of how to achieve this using async methods:

using System.Data.Entity;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Web;
using System.Web.Http;
using Newtonsoft.Json;

namespace StreamTest.Controllers
{
    public class DataController : ApiController
    {
        private readonly Data_EDM db = new Data_EDM();

        [Route("api/getdatastream")]
        public HttpResponseMessage GetDataStream(int count)
        {
            HttpResponseMessage response = Request.CreateResponse();

            response.Content = new PushStreamContent(
                async (outputStream, httpContent, transportContext) =>
                {
                    try
                    {
                        await db.ImportData.AsNoTracking().Take(count).ForEachAsync(async (entity) =>
                        {
                            var jsonString = JsonConvert.SerializeObject(entity);
                            var buffer = Encoding.UTF8.GetBytes(jsonString);

                            await outputStream.WriteAsync(buffer, 0, buffer.Length).ConfigureAwait(false);
                        }).ConfigureAwait(false);
                    }
                    catch (HttpException ex)
                    {
                        if (ex.ErrorCode == -2147023667) // The remote host closed the connection. 
                        {
                            return;
                        }
                    }
                    finally
                    {
                        // Close output stream as we are done
                        outputStream.Close();
                    }
                });
            return response;
        }
    }
}