A C# library to build RTSP Clients, RTSP Servers and handle RTP data streams. The library has several examples.

• RTSP Client Example - will connect to a RTSP server and receive Video and Audio in H264, H265/HEVC, G711, AAC and AMR formats. UDP, TCP and Multicast are supported. The data received is written to files.
• RTSP Camera Server Example - A YUV Image Generator and a very simple H264 Encoder generate H264 NALs which are then delivered via a RTSP Server to clients
• RTP Receiver - will recieve RTP and RTCP packets and pass them to a transport handler
• RTSP Server - will accept RTSP connections and talk to clients
• RTP Sender - will send RTP packets to clients
• Transport Handler - Transport hanlders for H264, H265/HEVC, G711 and AMR are provided.

⚠️ : This library does not handle the decoding of the video or audio (eg converting H264 into a bitmap). SharpRTSP is limited to the transport layer and generates the raw data that you need to feed into a video decoder or audio decoder. Many people use FFMPEG or use Hardware Accelerated Operating System APIs to do the decoding.

This is a walkthrough of an old version of the RTSP Client Example which highlights the main way to use the library.

• STEP 1 - Open TCP Socket connection to the RTSP Server

  // Connect to a RTSP Servertcp_socket=newRtsp.RtspTcpTransport(host,port);if(tcp_socket.Connected==false){Console.WriteLine("Error - did not connect");return;}

  This opens a connection for a 'TCP' mode RTSP/RTP session where RTP packets are set in the RTSP socket.

• STEP 2 - Create a RTSP Listener and attach it to the RTSP TCP Socket

  // Connect a RTSP Listener to the TCP Socket to send messages and listen for repliesrtsp_client=newRtsp.RtspListener(tcp_socket);rtsp_client.MessageReceived+=Rtsp_client_MessageReceived;rtsp_client.DataReceived+=Rtsp_client_DataReceived;rtsp_client.Start();// start reading messages from the server

  The RTSP Listener class lets you SEND messages to the RTSP Server (see below).
  The RTSP Listner class has a worker thread that listens for replies from the RTSP Server.
  When replies are received the MessageReceived Event is fired.
  When RTP packets are received the DataReceived Event is fired.

• STEP 3 - Send Messages to the RTSP Server

  The samples below show how to send messages.

  Send OPTIONS with this code :

  Rtsp.Messages.RtspRequestoptions_message=newRtsp.Messages.RtspRequestOptions();options_message.RtspUri=newUri(url);rtsp_client.SendMessage(options_message);

  Send DESCRIBE with this code :

  // send the DescribeRtsp.Messages.RtspRequestdescribe_message=newRtsp.Messages.RtspRequestDescribe();describe_message.RtspUri=newUri(url);rtsp_client.SendMessage(describe_message);// The reply will include the SDP data

  Send SETUP with this code :

  // the value of 'control' comes from parsing the SDP for the desired video or audio sub-streamRtsp.Messages.RtspRequestsetup_message=newRtsp.Messages.RtspRequestSetup();setup_message.RtspUri=newUri(url+"/"+control);setup_message.AddHeader("Transport: RTP/AVP/TCP;interleaved=0");rtsp_client.SendMessage(setup_message);// The reply will include the Session

  Send PLAY with this code :

  // the value of 'session' comes from the reply of the SETUP commandRtsp.Messages.RtspRequestplay_message=newRtsp.Messages.RtspRequestPlay();play_message.RtspUri=newUri(url);play_message.Session=session;rtsp_client.SendMessage(play_message);

• STEP 4 - Handle Replies when the MessageReceived event is fired

  This example assumes the main program sends an OPTIONS Command.
  It looks for a reply from the server for OPTIONS and then sends DESCRIBE.
  It looks for a reply from the server for DESCRIBE and then sends SETUP (for the video stream)
  It looks for a reply from the server for SETUP and then sends PLAY.
  Once PLAY has been sent the video, in the form of RTP packets, will be received.

  privatevoidRtsp_client_MessageReceived(objectsender,Rtsp.RtspChunkEventArgse){Rtsp.Messages.RtspResponsemessage=e.MessageasRtsp.Messages.RtspResponse;Console.WriteLine("Received "+message.OriginalRequest.ToString());if(message.OriginalRequest!=null&&message.OriginalRequestisRtsp.Messages.RtspRequestOptions){// send the DESCRIBERtsp.Messages.RtspRequestdescribe_message=newRtsp.Messages.RtspRequestDescribe();describe_message.RtspUri=newUri(url);rtsp_client.SendMessage(describe_message);}if(message.OriginalRequest!=null&&message.OriginalRequestisRtsp.Messages.RtspRequestDescribe){// Got a reply for DESCRIBE// Examine the SDPConsole.Write(System.Text.Encoding.UTF8.GetString(message.Data));Rtsp.Sdp.SdpFilesdp_data;using(StreamReadersdp_stream=newStreamReader(newMemoryStream(message.Data))){sdp_data=Rtsp.Sdp.SdpFile.Read(sdp_stream);}// Process each 'Media' Attribute in the SDP.// If the attribute is for Video, then send a SETUPfor(intx=0;x<sdp_data.Medias.Count;x++){if(sdp_data.Medias[x].GetMediaType()==Rtsp.Sdp.Media.MediaType.video){// seach the atributes for control, fmtp and rtpmapStringcontrol="";// the "track" or "stream id"Stringfmtp="";// holds SPS and PPSStringrtpmap="";// holds the Payload format, 96 is often used with H264foreach(Rtsp.Sdp.Attributattribinsdp_data.Medias[x].Attributs){if(attrib.Key.Equals("control"))control=attrib.Value;if(attrib.Key.Equals("fmtp"))fmtp=attrib.Value;if(attrib.Key.Equals("rtpmap"))rtpmap=attrib.Value;}// Get the Payload format number for the Video StreamString[]split_rtpmap=rtpmap.Split(' ');video_payload=0;boolresult=Int32.TryParse(split_rtpmap[0],outvideo_payload);// Send SETUP for the Video Stream// using Interleaved mode (RTP frames over the RTSP socket)Rtsp.Messages.RtspRequestsetup_message=newRtsp.Messages.RtspRequestSetup();setup_message.RtspUri=newUri(url+"/"+control);setup_message.AddHeader("Transport: RTP/AVP/TCP;interleaved=0");rtsp_client.SendMessage(setup_message);}}}if(message.OriginalRequest!=null&&message.OriginalRequestisRtsp.Messages.RtspRequestSetup){// Got Reply to SETUPConsole.WriteLine("Got reply from Setup. Session is "+message.Session);Stringsession=message.Session;// Session value used with Play, Pause, Teardown// Send PLAYRtsp.Messages.RtspRequestplay_message=newRtsp.Messages.RtspRequestPlay();play_message.RtspUri=newUri(url);play_message.Session=session;rtsp_client.SendMessage(play_message);}if(message.OriginalRequest!=null&&message.OriginalRequestisRtsp.Messages.RtspRequestPlay){// Got Reply to PLAYConsole.WriteLine("Got reply from Play "+message.Command);}}

• STEP 5 - Handle RTP Video

  This code handles each incoming RTP packet, combining RTP packets that are all part of the same frame of vdeo (using the Marker Bit). Once a full frame is received it can be passed to a De-packetiser to get the compressed video data

  List<byte[]>temporary_rtp_payloads=newList<byte[]>();privatevoidRtsp_client_DataReceived(objectsender,Rtsp.RtspChunkEventArgse){// RTP Packet Header// 0 - Version, P, X, CC, M, PT and Sequence Number//32 - Timestamp//64 - SSRC//96 - CSRCs (optional)//nn - Extension ID and Length//nn - Extension headerintrtp_version=(e.Message.Data[0]>>6);intrtp_padding=(e.Message.Data[0]>>5)&0x01;intrtp_extension=(e.Message.Data[0]>>4)&0x01;intrtp_csrc_count=(e.Message.Data[0]>>0)&0x0F;intrtp_marker=(e.Message.Data[1]>>7)&0x01;intrtp_payload_type=(e.Message.Data[1]>>0)&0x7F;uintrtp_sequence_number=((uint)e.Message.Data[2]<<8)+(uint)(e.Message.Data[3]);uintrtp_timestamp=((uint)e.Message.Data[4]<<24)+(uint)(e.Message.Data[5]<<16)+(uint)(e.Message.Data[6]<<8)+(uint)(e.Message.Data[7]);uintrtp_ssrc=((uint)e.Message.Data[8]<<24)+(uint)(e.Message.Data[9]<<16)+(uint)(e.Message.Data[10]<<8)+(uint)(e.Message.Data[11]);intrtp_payload_start=4// V,P,M,SEQ+4// time stamp+4// ssrc+(4*rtp_csrc_count);// zero or more csrcsuintrtp_extension_id=0;uintrtp_extension_size=0;if(rtp_extension==1){rtp_extension_id=((uint)e.Message.Data[rtp_payload_start+0]<<8)+(uint)(e.Message.Data[rtp_payload_start+1]<<0);rtp_extension_size=((uint)e.Message.Data[rtp_payload_start+2]<<8)+(uint)(e.Message.Data[rtp_payload_start+3]<<0);rtp_payload_start+=4+(int)rtp_extension_size;// extension header and extension payload}Console.WriteLine("RTP Data"+" V="+rtp_version+" P="+rtp_padding+" X="+rtp_extension+" CC="+rtp_csrc_count+" M="+rtp_marker+" PT="+rtp_payload_type+" Seq="+rtp_sequence_number+" Time="+rtp_timestamp+" SSRC="+rtp_ssrc+" Size="+e.Message.Data.Length);if(rtp_payload_type!=video_payload){Console.WriteLine("Ignoring this RTP payload");return;// ignore this data}// If rtp_marker is '1' then this is the final transmission for this packet.// If rtp_marker is '0' we need to accumulate data with the same timestamp// ToDo - Check Timestamp matches// Add to the tempoary_rtp Listbyte[]rtp_payload=newbyte[e.Message.Data.Length-rtp_payload_start];// payload with RTP header removedSystem.Array.Copy(e.Message.Data,rtp_payload_start,rtp_payload,0,rtp_payload.Length);// copy payloadtemporary_rtp_payloads.Add(rtp_payload);if(rtp_marker==1){// Process the RTP frameProcess_RTP_Frame(temporary_rtp_payloads);temporary_rtp_payloads.Clear();}}

• STEP 6 - Process RTP frame

  An RTP frame consists of 1 or more RTP packets
  H264 video is packed into one or more RTP packets and this sample extracts Normal Packing and Fragmented Unit type A packing (the common two)
  This example writes the video to a .264 file which can be played with FFPLAY

  FileStreamfs=null;byte[]nal_header=newbyte[]{0x00,0x00,0x00,0x01};intnorm,fu_a,fu_b,stap_a,stap_b,mtap16,mtap24=0;// stats counterspublicvoidProcess_RTP_Frame(List<byte[]>rtp_payloads){Console.WriteLine("RTP Data comprised of "+rtp_payloads.Count+" rtp packets");if(fs==null){// Create the fileStringfilename="rtsp_capture_"+DateTime.Now.ToString("yyyyMMdd_HHmmss")+".h264";fs=newFileStream(filename,FileMode.Create);// TODO. Get SPS and PPS from the SDP Attributes (the fmtp attribute) and write to the file// for IP cameras that only out the SPS and PPS out-of-band}for(intpayload_index=0;payload_index<rtp_payloads.Count;payload_index++){// Examine the first rtp_payload and the first byte (the NAL header)intnal_header_f_bit=(rtp_payloads[payload_index][0]>>7)&0x01;intnal_header_nri=(rtp_payloads[payload_index][0]>>5)&0x03;intnal_header_type=(rtp_payloads[payload_index][0]>>0)&0x1F;// If the NAL Header Type is in the range 1..23 this is a normal NAL (not fragmented)// So write the NAL to the fileif(nal_header_type>=1&&nal_header_type<=23){Console.WriteLine("Normal NAL");norm++;fs.Write(nal_header,0,nal_header.Length);fs.Write(rtp_payloads[payload_index],0,rtp_payloads[payload_index].Length);}elseif(nal_header_type==24){// There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)Console.WriteLine("Agg STAP-A not supported");stap_a++;}elseif(nal_header_type==25){// There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)Console.WriteLine("Agg STAP-B not supported");stap_b++;}elseif(nal_header_type==26){// There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)Console.WriteLine("Agg MTAP16 not supported");mtap16++;}elseif(nal_header_type==27){// There are 4 types of Aggregation Packet (multiple NALs in one RTP packet)Console.WriteLine("Agg MTAP24 not supported");mtap24++;}elseif(nal_header_type==28){Console.WriteLine("Fragmented Packet Type FU-A");fu_a++;// Parse Fragmentation Unit Headerintfu_header_s=(rtp_payloads[payload_index][1]>>7)&0x01;// start markerintfu_header_e=(rtp_payloads[payload_index][1]>>6)&0x01;// end markerintfu_header_r=(rtp_payloads[payload_index][1]>>5)&0x01;// reserved. should be 0intfu_header_type=(rtp_payloads[payload_index][1]>>0)&0x1F;// Original NAL unit headerConsole.WriteLine("Frag FU-A s="+fu_header_s+"e="+fu_header_e);// Start Flag setif(fu_header_s==1){// Write 00 00 00 01 headerfs.Write(nal_header,0,nal_header.Length);// 0x00 0x00 0x00 0x01// Modify the NAL Header that was at the start of the RTP packet// Keep the F and NRI flags but substitute the type field with the fu_header_typebytereconstructed_nal_type=(byte)((nal_header_nri<<5)+fu_header_type);fs.WriteByte(reconstructed_nal_type);// NAL Unit Typefs.Write(rtp_payloads[payload_index],2,rtp_payloads[payload_index].Length-2);// start after NAL Unit Type and FU Header byte}if(fu_header_s==0){// append this payload to the output NAL stream// Data starts after the NAL Unit Type byte and the FU Header bytefs.Write(rtp_payloads[payload_index],2,rtp_payloads[payload_index].Length-2);// start after NAL Unit Type and FU Header byte}}elseif(nal_header_type==29){Console.WriteLine("Fragmented Packet FU-B not supported");fu_b++;}else{Console.WriteLine("Unknown NAL header "+nal_header_type);}}// ensure video is written to diskfs.Flush(true);// Print totalsConsole.WriteLine("Norm="+norm+" ST-A="+stap_a+" ST-B="+stap_b+" M16="+mtap16+" M24="+mtap24+" FU-A="+fu_a+" FU-B="+fu_b);}