Partial and asynchronous deserialization c# with protobuf-net -

context

i'm have file following structure:

[protocontract] public class data {     [protomember(1)]     public string header { get; set; }      [protomember(2)]     public byte[] body { get; set; } } 

the code reads / writes data file running on asp.net mvc webapi context. i'm trying keep every blocking io async minimize blocking , achieve best scalability. reading , writing files support readasync, writeasync , copytoasync.

the body can reasonably large (>> header) , need read body if header matches specific criteria.

i can partially read , deserialize header synchronously , read , deserialize body same way using approach explained in deserialize part of binary file

problem

how can use asynchronous file io same, reading , deserializing header async , reading , deserializing body same way?

i've read asynchronous protobuf serialization not option.

technically protobuf fields can out-of-order, in cases (including 1 show) can reasonable assume fields in-order (the way them out-of-order here separately serialize 2 half-classes , concatenate results, technically valid in protobuf specification).

so; have is:

• a varint denoting: field 1, string - decimal 10
• a varint denoting "a", length of header
• "a" bytes, utf-8 encoded header
• a varint denoting: field 2, string - decimal 18
• a varint denoting "b", length of body
• "b" bytes, body

we can assume "a" >= 0 , < int.maxvalue - means take @ 5 bytes encode; so, if buffer at least 6 bytes, have enough information know how large header is. of course, technically contain part of body, you'd need keep hold of it! if had sync-over-async stream, can read just part of stream like:

int protoheader = protoreader.directreadvarintint32(stream); // 10 int headerlength = protoreader.directreadvarintint32(stream); string header = protoreader.directreadstring(stream, headerlength); 

or if "sync on async" tricky, explicit reading:

static byte[] readatleast6() {     return new byte[] { 0x0a, 0x0b, 0x68, 0x65, 0x6c, 0x6c, 0x6f }; } static byte[] readmore(int bytes) {     return new byte[] { 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64 }; } static void main() {     // pretend read 7 bytes async     var data = readatleast6();     using (var ms = new memorystream())     {         ms.write(data, 0, data.length);         ms.position = 0;         int protoheader = protoreader.directreadvarintint32(ms); // 10         int headerlength = protoreader.directreadvarintint32(ms); // 11          int needed = (headerlength + (int)ms.position) - data.length; // 6 more         var pos = ms.position;         ms.seek(0, seekorigin.end);         data = readmore(needed);         ms.write(data, 0, needed);         ms.position = pos;         string header = protoreader.directreadstring(ms, headerlength);     } }