21 April 2023

How We Implemented User Generated Video Recording And Playback In Perxeive

Perxeive is an iOS app for iPhone and iPad that is distributed via the Apple App Store and which enables users to create, edit and share factual content. One of the features of Perxeive is the ability to record, edit, upload and share short videos. In this post we discuss how we overcame one of the key challenges of delivering the video feature to meet Apple's strict standards on video playback.

The Challenge Of Delivering Video Playback On iOS Devices

A key feature of Perxeive is to provide users with the ability to record, edit, upload and share short videos. Apple imposes strict requirements on developers with respect to the video file encoding that can be played on the native iOS API video player. In particular, Apple requires developers to stream video on iOS devices using only its own HLS video encoding standard.

This requirement is to ensure high quality video streaming on Apple devices for a range of data bandwidth scenarios. Essentially, HLS video encoding breaks the video down into chunks that can be streamed from a remote server sequentially. The video can be created in a range of chunk sizes to optimise for low to high data bandwidth scenarios. HLS delivers a superior user experience by avoiding the need for an entire video to be downloaded before playback can commence as would be required for, say, an MP4 encoded video.

The challenge in delivering the video features in Perxeive was to enable users to upload videos encoded in a number of formats and to then transcode those videos into HLS for playback on Apple devices.

Implementing AWS MediaConvert

Having researched a number of options for transcoding the various video codecs into HLS we decided to implement AWS MediaConvert in our backend server set up. AWS MediaConvert is a cloud-based video transcoding service offered by Amazon Web Services (AWS) that enables users to convert their video files into multiple formats, resolutions, and codecs for seamless playback across various devices, platforms, and browsers.

Since we were already hosting our servers in AWS Fargate in a VPC and were also storing user generated content is AWS S3 buckets, AWS MediaConvert provided the fastest time to market whilst meeting our requirements for video encoding flexibility.

The Benefits Of Implementing AWS MediaConvert

1. Faster Time-to-Market: MediaConvert accelerates the video transcoding process, enabling faster content delivery to audiences across various platforms.

2. Improved Video Quality: The service ensures high-quality video output, adapting to different devices, networks, and playback environments.

3. Cost-Effective: MediaConvert provides a scalable, on-demand pricing model, reducing the need for upfront capital expenditures or ongoing maintenance costs.

4. Increased Flexibility: Users can process videos in multiple formats and resolutions, catering to diverse audience needs and preferences.

5. Scalability: MediaConvert can handle large volumes of video content, making it an ideal solution for enterprises, media companies, and content creators.

6. Format Support: The service supports a wide range of input formats, including MP4, MOV, AVI, and more, and can output files in popular formats like HLS, DASH, MP4, WebM, and others.

7. Automated Workflows: MediaConvert allows users to create custom workflows using APIs, SDKs, or the AWS Management Console, enabling automated video processing and transcoding.

8. Quality Control: The service provides advanced quality control features, including multi-pass encoding, dynamic content adaptation, and quality-based encoding to ensure high-quality output.

9. Security and Compliance: MediaConvert follows AWS's robust security and compliance standards, ensuring the secure handling and processing of video content.

10. Integration with AWS Services: MediaConvert integrates seamlessly with other AWS services like Amazon S3, Amazon CloudFront, and AWS Elemental MediaStore for a comprehensive media workflow.

Implementation Steps For AWS MediaConvert

1. Set up an AWS S3 bucket to store uploaded videos

Setting up the S3 bucket is straightforward. We followed the advice from the implementation examples we reviewed online and named the bucket with a 'watchfolder' suffix. The only other implementation detail to mention was the video file naming convention that we chose was to use random UUIDs plus the video format extension.

Access to the S3 bucket was limited via a security policy such that only AWS Cognito authenticated app users could PUT files and only the AWS Lambda function could GET files.

Another implementation detail that was specific to our set up was the need to include an AWS IAM Role to enable the uploaded videos to be written to the S3 bucket via AWS API Gateway.

2. Set up an AWS S3 bucket to store videos converted to HLS format

Again, setting up the S3 bucket to store the converted videos was straightforward. The bucket name was given the 'output' suffix to easily distinguish it from the 'watchfolder' described above.

Access to the S3 bucket was limited via a security policy such that only the AWS Lambda function had PUT permissions and only AWS Cognito authenticated app users had GET permissions.

The output bucket also required a CORS configuration to be implemented.

3. Create an AWS IAM Role to provide AWS MediaConvert with the necessary permissions

A critical step in the implementation of AWS MediaConvert is the set up of an AWS IAM Role for MediaConvert. The role granted permissions relating to the S3 bucket access and API Gateway access.

4. Create an AWS IAM Role to provide an AWS Lambda function with the necessary permissions

The AWS IAM Role that was created for the AWS Lambda function was a combination of the default basic execution role for AWS Lambda and permissions specific to AWS MediaConvert.

5. Implement an AWS Lambda function to perform the video transcoding

The most significant effort in implementing AWS MediaConvert was the implementation of the AWS Lambda function. This required a large amount of trial and error as none of the example implementations in the various tutorials available online provided working examples for our specific use case.

The documentation available from AWS and the examples provided there did not provide what we needed to successfully implement MediaConvert. There are so many permutations of specific configuration parameters and to meet our own very specific use case required a huge amount of persistence.

There are two components to the AWS Lambda function that manages the process of transcoding videos using AWS MediaConvert. First there is a python script that handles getting the uploaded video file from the S3 watchfolder bucket and then invokes MediaConvert to perform the video transcoding process. The second component is a JSON file named job.json that provides the configuration parameters to MediaConvert to be able to perform the specific transcoding that you reqiure.

The python file was pretty straightforward. The examples given in the official documentation can easily be adapted to the specifics of your requirements.

The JSON configuration was much more challenging to implement. The examples given in the official documentation and also all the examples found online failed to work for our use case, despite the use case being standard. There were specific 'DestinationSettings' included in all examples that needed to be removed. Also, the 'OutputGroupSettings' that were required differed materially from those found in the official documents or in online examples. The time it took to figure this out was considerable and the process frustrating. Whilst the official documentation of the MediaConvert configuration options is both detailed and extensive, the lack of practical examples of JSON based configurations meant that trial and error was the only viable option.

6. Create an AWS S3 Event Trigger to trigger the AWS Lambda when a video is uploaded

To complete the implementation of the video transcoding it was necessary to set up an Event Trigger on the AWS S3 'watchfolder' bucket to trigger the AWS Lambda function when a new video was uploaded.

Conclusion

To provide users of the Perxeive app with the ability to record, edit, upload and share short videos it was necessary to implement a video transcoding solution that satisfied Apple's video playback technical requirements. Specifically, it was necessary to be able to upload videos in various video formats and to transcode them to Apple's HLS format in order for the videos to play in the native video player of the iOS API.

AWS MediaConvert was successfully implemented. It provides a robust, flexible, scalable and low cost solution to our requirements.

---