Audio Streaming and Caching in iOS using AVAssetResourceLoader and AVPlayer



In this post you will learn how to use AVAssetResourceLoaderDelegate with example app.

A week ago we published Evermusic 1.5 and now our cloud player can stream music from Yandex.Disk and Web Dav as well as from DropBox and Google Drive. If you want to know how we implemented new services in Evermusic and how you can use AVAssetResourceLoader and AVPlayer in your apps this tutorial is for you.

We use AVPlayer for playing audio files from local storage and remote host. If you want to stream an audio file from some cloud service you should get a direct url of that file and initialize AVPlayer with received url. We use this approach for the most cloud services like DropBox, Box, Google Drive but it does not work when you play audio files from Yandex.Disk and Web Dav. It’s all because of authorization header that you should set when you send GET request to the server. There is no public method in AVPlayer interface that allows you to do this.

So we started looking for a solution and came across the resourceLoader object in AVURLAsset. This is actually a great API and you can use it to provide controlled access to the remote file for AVPlayer. This works like a local HTTP proxy but without all the hassles.

The most important thing to remember is that AVPlayer uses resourceLoader when it doesn’t know how to load a resource. The trick here is to change the protocol so AVPlayer is forced to defer the loading of the resource to our application. So we should change scheme of our resource and initialize AVPlayer with new url.

When you work with AVAssetResourceLoader you should implement two methods of AVAssetResourceLoaderDelegate:

- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest;

- (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest;

Delegates receive resourceLoader: shouldWaitForLoadingOfRequestedResource: message when assistance is required of the application to load a resource. In this case we save AVAssetResourceLoadingRequest and start data loading operation. When data from the resource is no longer required or when a loading request is superseded by new requests for data from the same resource, delegates receive resourceLoader: didCancelLoadingRequest: and we cancel data loading operation.

So let’s create our AVPlayer with custom scheme:

NSURL *url = [NSURL URLWithString:@"customscheme://host/audio.mp3"]; AVURLAsset *asset = [AVURLAsset URLAssetWithURL:url options:nil]; [asset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()]; AVPlayerItem *item = [AVPlayerItem playerItemWithAsset:asset];

[self addObserversForPlayerItem:item];

self.player = [AVPlayer playerWithPlayerItem:playerItem];

[self addObserversForPlayer];

The first two lines create AVURLAsset with custom url and set AVAssetResourceLoaderDelegate with dispatch queue on which delegate methods will be invoked.

The next two lines create AVPlayerItem from AVURLAsset and AVPlayer from AVPlayerItem and add required observers.

The next thing we should do is create custom class that will load data of requested resource from the server and pass loaded data back to AVURLAsset. Let’s name it LSFilePlayerResourceLoader and add two parameters in init method. The first parameter is requested file url and the second is YDSession object. This session object is responsible for getting file data from Yandex.Disk cloud server.

Our LSFilePlayerResourceLoader objects will be stored in dictionary and resource url will be the key.

Here’s what our AVAssetResourceLoaderDelegate implementation will looks like:

- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest{

NSURL *resourceURL = [loadingRequest.request URL];

if([resourceURL.scheme isEqualToString:@"customscheme"]){ LSFilePlayerResourceLoader *loader = [self resourceLoaderForRequest:loadingRequest];

if(loader==nil){

loader = [[LSFilePlayerResourceLoader alloc] initWithResourceURL:resourceURL session:self.session];

loader.delegate = self;

[self.resourceLoaders setObject:loader forKey:[self keyForResourceLoaderWithURL:resourceURL]]; }

[loader addRequest:loadingRequest];

return YES; }

return NO; }

- (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest{ LSFilePlayerResourceLoader *loader = [self resourceLoaderForRequest:loadingRequest];

[loader removeRequest:loadingRequest]; }

First we should check resourceURL scheme and then get cached LSFilePlayerResourceLoader or create the new one. After this we add loadingRequest to our resource loader.

LSFilePlayerResourceLoader interface is below:

@interface LSFilePlayerResourceLoader : NSObject

@property (nonatomic,readonly,strong)NSURL *resourceURL;

@property (nonatomic,readonly)NSArray *requests;

@property (nonatomic,readonly,strong)YDSession *session;

@property (nonatomic,readonly,assign)BOOL isCancelled;

@property (nonatomic,weak)id<LSFilePlayerResourceLoaderDelegate> delegate;

- (instancetype)initWithResourceURL:(NSURL *)url session:(YDSession *)session;

- (void)addRequest:(AVAssetResourceLoadingRequest *)loadingRequest;

- (void)removeRequest:(AVAssetResourceLoadingRequest *)loadingRequest;

- (void)cancel;

@end

@protocol LSFilePlayerResourceLoaderDelegate <NSObject>

@optional

- (void)filePlayerResourceLoader:(LSFilePlayerResourceLoader *)resourceLoader didFailWithError:(NSError *)error;

- (void)filePlayerResourceLoader:(LSFilePlayerResourceLoader *)resourceLoader didLoadResource:(NSURL *)resourceURL;

@end

This interface has methods for managing requests in loader queue and LSFilePlayerResourceLoaderDelegate protocol which defines methods that allow your code to handle resource loading status.

When we add loadingRequest to the queue we save it in pendingRequests array and start data loading operation:

- (void)addRequest:(AVAssetResourceLoadingRequest *)loadingRequest{ if(self.isCancelled==NO){

NSURL *interceptedURL = [loadingRequest.request URL];

[self startOperationFromOffset:loadingRequest.dataRequest.requestedOffset length:loadingRequest.dataRequest.requestedLength];

[self.pendingRequests addObject:loadingRequest];

}

else{

if(loadingRequest.isFinished==NO){

[loadingRequest finishLoadingWithError:[self loaderCancelledError]];

} } }

First time we created a new data load operation for every upcoming request. That’s why the file was loaded from multiple threads. But then we found out that when AVAssetResourceLoader began new loading request previously issued requests can be cancelled. So in our start operation method we cancel all previously started operations .

There are two data loading operations. A contentInfoOperation is operation to identify the content length, content type, and whether the resource supports byte range requests. With byte range requests, AVPlayer can get fancy and apply various optimizations. The second one is dataOperation which loads file data with offset. We get requested data offset from AVAssetResourceLoadingDataRequest.

- (void)startOperationFromOffset:(unsigned long long)requestedOffset length:(unsigned long long)requestedLength{

[self cancelAllPendingRequests];

[self cancelOperations];

__weak typeof (self) weakSelf = self;

void(^failureBlock)(NSError *error) = ^(NSError *error) {

[weakSelf performBlockOnMainThreadSync:^{

if(weakSelf && weakSelf.isCancelled==NO){

[weakSelf completeWithError:error]; } }]; };

void(^loadDataBlock)(unsigned long long off, unsigned long long len) = ^(unsigned long long offset,unsigned long long length){

[weakSelf performBlockOnMainThreadSync:^{

NSString *bytesString = [NSString stringWithFormat:@"bytes=%lld-%lld",offset,(offset+length-1)];

NSDictionary *params = @{@"Range":bytesString};

id<YDSessionRequest> req = [weakSelf.session partialContentForFileAtPath:weakSelf.path withParams:params response:nil data:^(UInt64 recDataLength, UInt64 totDataLength, NSData *recData) {

[weakSelf performBlockOnMainThreadSync:^{

if(weakSelf && weakSelf.isCancelled==NO){

LSDataResonse *dataResponse = [LSDataResonse responseWithRequestedOffset:offset requestedLength:length receivedDataLength:recDataLength data:recData];

[weakSelf didReceiveDataResponse:dataResponse];

} }]; }

completion:^(NSError *err) { if(err){ failureBlock(err); } }];

weakSelf.dataOperation = req; }]; };

if(self.contentInformation==nil){

self.contentInfoOperation = [self.session fetchStatusForPath:self.path completion:^(NSError *err, YDItemStat *item) {

if(weakSelf && weakSelf.isCancelled==NO){ if(err==nil){ NSString *mimeType = item.path.mimeTypeForPathExtension; CFStringRef contentType = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType,(__bridge CFStringRef)(mimeType),NULL);

unsigned long long contentLength = item.size;

weakSelf.contentInformation = [[LSContentInformation alloc] init]; weakSelf.contentInformation.byteRangeAccessSupported = YES; weakSelf.contentInformation.contentType = CFBridgingRelease(contentType); weakSelf.contentInformation.contentLength = contentLength;

[weakSelf prepareDataCache];

loadDataBlock(requestedOffset,requestedLength);

weakSelf.contentInfoOperation = nil; }

else{ failureBlock(err); } } }]; } else{ loadDataBlock(requestedOffset,requestedLength); } }

When the contentInformation request is received and no cached file exists, we init data cache and start fetching the audio file.

We use temporary file for caching received data from the web and read it when we need.

- (void)prepareDataCache{ self.cachedFilePath = [[self class] pathForTemporaryFile]; NSError *error = nil; if ([[NSFileManager defaultManager] fileExistsAtPath:self.cachedFilePath] == YES){ [[NSFileManager defaultManager] removeItemAtPath:self.cachedFilePath error:&error]; }

if (error == nil && [[NSFileManager defaultManager] fileExistsAtPath:self.cachedFilePath] == NO) {

NSString *dirPath = [self.cachedFilePath stringByDeletingLastPathComponent]; [[NSFileManager defaultManager] createDirectoryAtPath:dirPath withIntermediateDirectories:YES attributes:nil error:&error];

if (error == nil) { [[NSFileManager defaultManager] createFileAtPath:self.cachedFilePath contents:nil attributes:nil];

self.writingFileHandle = [NSFileHandle fileHandleForWritingAtPath:self.cachedFilePath]; @try { [self.writingFileHandle truncateFileAtOffset:self.contentInformation.contentLength];

[self.writingFileHandle synchronizeFile]; }

@catch (NSException *exception) { NSError *error = [[NSError alloc] initWithDomain: LSFilePlayerResourceLoaderErrorDomain code: -1 userInfo: @{ NSLocalizedDescriptionKey : @"can not write to file" }];

[self completeWithError:error]; return; }

self.readingFileHandle = [NSFileHandle fileHandleForReadingAtPath:self.cachedFilePath]; } }

if (error != nil) { [self completeWithError:error]; } }

When new data is received we cache it on the disk, update receivedDataLength and then notify all pending requests.

- (void)didReceiveDataResponse:(LSDataResonse *)dataResponse{ [self cacheDataResponse:dataResponse]; self.receivedDataLength=dataResponse.currentOffset; [self processPendingRequests]; }

Cache data response method is responsible for caching received data with requested offset.

- (void)cacheDataResponse:(LSDataResonse *)dataResponse{ unsigned long long offset = dataResponse.dataOffset;

@try { [self.writingFileHandle seekToFileOffset:offset];

[self.writingFileHandle writeData:dataResponse.data];

[self.writingFileHandle synchronizeFile]; }

@catch (NSException *exception) {

NSError *error = [[NSError alloc] initWithDomain: LSFilePlayerResourceLoaderErrorDomain code: -1 userInfo: @{ NSLocalizedDescriptionKey : @"can not write to file" }];

[self completeWithError:error]; } }

Read data method is responsible for reading cached data from disk.

- (NSData *)readCachedData:(unsigned long long)startOffset length:(unsigned long long)numberOfBytesToRespondWith{ @try { [self.readingFileHandle seekToFileOffset:startOffset]; NSData *data = [self.readingFileHandle readDataOfLength:numberOfBytesToRespondWith]; return data; } @catch (NSException *exception) {} return nil; }

In processPendingRequests method we fill content information and write cached data. When all requested data is received we remove pending request from queue.

- (void)processPendingRequests{ NSMutableArray *requestsCompleted = [[NSMutableArray alloc] init];

for (AVAssetResourceLoadingRequest *loadingRequest in self.pendingRequests){

[self fillInContentInformation:loadingRequest.contentInformationRequest];

BOOL didRespondCompletely = [self respondWithDataForRequest:loadingRequest.dataRequest];

if (didRespondCompletely){ [loadingRequest finishLoading];

[requestsCompleted addObject:loadingRequest]; } }

[self.pendingRequests removeObjectsInArray:requestsCompleted]; }

Write information about content such as content length, content type, and whether the resource supports byte range requests.

- (void)fillInContentInformation:(AVAssetResourceLoadingContentInformationRequest *) contentInformationRequest{ if (contentInformationRequest == nil || self.contentInformation == nil){ return; } contentInformationRequest.byteRangeAccessSupported = self.contentInformation.byteRangeAccessSupported; contentInformationRequest.contentType = self.contentInformation.contentType; contentInformationRequest.contentLength = self.contentInformation.contentLength; }

Read data from cache and pass it to pending requests.

- (BOOL)respondWithDataForRequest:(AVAssetResourceLoadingDataRequest *)dataRequest{ long long startOffset = dataRequest.requestedOffset;

if (dataRequest.currentOffset != 0){ startOffset = dataRequest.currentOffset; }

// Don't have any data at all for this request

if (self.receivedDataLength < startOffset){ return NO; }

// This is the total data we have from startOffset to whatever has been downloaded so far

NSUInteger unreadBytes = self.receivedDataLength - startOffset;

// Respond with whatever is available if we can't satisfy the request fully yet

NSUInteger numberOfBytesToRespondWith = MIN(dataRequest.requestedLength, unreadBytes);

BOOL didRespondFully = NO;

NSData *data = [self readCachedData:startOffset length:numberOfBytesToRespondWith];

if(data){

[dataRequest respondWithData:data];

long long endOffset = startOffset + dataRequest.requestedLength;

didRespondFully = self.receivedDataLength >= endOffset; }

return didRespondFully; }

That’s all with our loader. It’s time to patch Yandex.Disk SDK and add method for getting partial content from server. In you case it might be different cloud service with it’s own SDK but you should apply the same changes as described below. There are only 3 of them.

The first we should check if all requests in given SDK has cancel method. Unfortunately client code can not cancel any request in Yandex.Disk SDK so we should add this possibility. Just declare new protocol YDSessionRequest in YDSession.h and return it in all requests.

@protocol YDSessionRequest <NSObject> - (void)cancel; @end - (id<YDSessionRequest>)fetchDirectoryContentsAtPath:(NSString *)path completion:(YDFetchDirectoryHandler)block;

- (id<YDSessionRequest>)fetchStatusForPath:(NSString *)path completion:(YDFetchStatusHandler)block;

The next we should implement method for getting partial data from requested file with given offset and length.

- (id<YDSessionRequest>)partialContentForFileAtPath:(NSString *)srcRemotePath withParams:(NSDictionary *)params response:(YDDidReceiveResponseHandler)response data:(YDPartialDataHandler)data completion:(YDHandler)completion{

return [self downloadFileFromPath:srcRemotePath toFile:nil withParams:params response:response data:data progress:nil completion:completion]; }

- (id<YDSessionRequest>)downloadFileFromPath:(NSString *)path toFile:(NSString *)aFilePath withParams:(NSDictionary *)params response:(YDDidReceiveResponseHandler)responseBlock data:(YDPartialDataHandler)dataBlock progress:(YDProgressHandler)progressBlock completion:(YDHandler)completionBlock{

NSURL *url = [YDSession urlForDiskPath:path];

if (!url) { completionBlock([NSError errorWithDomain: kYDSessionBadArgumentErrorDomain code: 0 userInfo: @{ @"getPath": path }]);

return nil; }

BOOL skipReceivedData = NO;

if(aFilePath==nil){ aFilePath = [[self class] pathForTemporaryFile];

skipReceivedData = YES; }

NSURL *filePath = [YDSession urlForLocalPath:aFilePath];

if (!filePath) { completionBlock([NSError errorWithDomain: kYDSessionBadArgumentErrorDomain code: 1 userInfo: @{ @"toFile": aFilePath }]); return nil; }

YDDiskRequest *request = [[YDDiskRequest alloc] initWithURL:url];

request.fileURL = filePath;

request.params = params;

request.skipReceivedData = skipReceivedData;

[self prepareRequest:request];

NSURL *requestURL = [request.URL copy];

request.callbackQueue = _callBackQueue; request.didReceiveResponseBlock = ^(NSURLResponse *response, BOOL *accept) { if(responseBlock){ responseBlock(response); } }; request.didGetPartialDataBlock = ^(UInt64 receivedDataLength, UInt64 expectedDataLength, NSData *data){ if(progressBlock){ progressBlock(receivedDataLength,expectedDataLength); } if(dataBlock){ dataBlock(receivedDataLength,expectedDataLength,data); } }; request.didFinishLoadingBlock = ^(NSData *receivedData) { if(skipReceivedData){ [[self class] removeTemporaryFileAtPath:aFilePath]; } NSDictionary *userInfo = @{@"URL": requestURL, @"receivedDataLength": @(receivedData.length)}; [[NSNotificationCenter defaultCenter] postNotificationInMainQueueWithName: kYDSessionDidDownloadFileNotification object: self userInfo: userInfo];

completionBlock(nil); };

request.didFailBlock = ^(NSError *error) { if(skipReceivedData){ [[self class] removeTemporaryFileAtPath:aFilePath]; } NSDictionary *userInfo = @{@"URL": requestURL};

[[NSNotificationCenter defaultCenter] postNotificationInMainQueueWithName: kYDSessionDidFailToDownloadFileNotification object: self userInfo: userInfo]; completionBlock([NSError errorWithDomain:error.domain code:error.code userInfo:userInfo]); }; [request start];

NSDictionary *userInfo = @{@"URL": request.URL}; [[NSNotificationCenter defaultCenter] postNotificationInMainQueueWithName: kYDSessionDidStartDownloadFileNotification object: self userInfo: userInfo]; return (id<YDSessionRequest>)request; }

And the last but not least we should check if our Yandex.Disk SDK has serial callback queue. It’s very important to use serial queue because playback issues may occur if parallel queue is used. In our case Yandex.Disk SDK has parallel callback queue by default so let’s fix this.

- (instancetype)initWithDelegate:(id<YDSessionDelegate>)delegate callBackQueue:(dispatch_queue_t)queue{

self = [super init];

if (self) { _delegate = delegate;

_callBackQueue = queue; }

return self; }

YDDiskRequest *request = [[YDDiskRequest alloc] initWithURL:url];

request.fileURL = filePath;

request.params = params;

[self prepareRequest:request];

request.callbackQueue = _callBackQueue;

The source code given in this tutorial is available on GitHub.

#AVAssetResourceLoader #AVPlayer #AVURLAsset #AVAssetResourceLoadingRequest #streaming #iOS #SDK #AVFoundation #Framework

3,339 views
Company
Please reload

Products
Please reload

  • Facebook - White Circle
  • Twitter - White Circle
  • Vkontakte - White Circle
Copyright © 2020 everappz.com