This post will show you some strategies that you can use for such a situation. In the first part of this three-part series, we introduced two basic strategies for dealing with data when your app needs to work offline: online-first and offline-first. This second part focuses on the online-first strategy.

Table of contents

The online-first strategy

The online-first strategy is the obvious choice when you have an existing application to which you want to add offline capabilities. It is also a good strategy to use when you are building a new app that only needs offline capabilities for a very limited, specific part of the app. To implement such a strategy, you essentially build your app as if it is a regular on-line only app, and then you add some magic fairy dust that will make certain parts offline-capable.

So what does this magic fairy dust look like?

Turn your app into a PWA / The power of service workers

Despite my tongue-in-cheek remark about PWAs in the intro to this post, this is actually a very good starting point. If you want to add offline capabilities to your app, turning it into a PWA is a good start. If you simply follow Angular’s basic guidelines, you will get the foundational pieces that you need, the most important of which: a service worker implementation.

A service worker is a special piece of JavaScript. As MDN puts it:

Service workers essentially act as proxy servers that sit between web applications, the browser, and the network (when available). They are intended, among other things, to enable the creation of effective offline experiences, intercept network requests and take appropriate action based on whether the network is available, and update assets residing on the server. They will also allow access to push notifications and background sync APIs.

Or, as summarized by Angular:

At its simplest, a service worker is a script that runs in the web browser and manages caching for an application.

Service workers make it possible to create an “effective offline experience”, so how do we do that?

When working with data, there are typically two scenarios that we need to deal with: retrieving the data, such that is available when we need it, and saving the data, such that any changes to it are persisted.

Retrieving data

If you want to work with data when your are offline, you obviously need to have retrieved it beforehand, when you were still online. The service worker can help with this.

The basic recipe

The basic recipe for this strategy is as follows:

  1. When you are still online, retrieve the data that you need to have available when offline.
  2. Configure the service worker such that it will cache the retrieved data.
  3. Now when you are offline, you can simply try and retrieve the data as if you were online, and the service worker will provide the cached data to your app as if you had just retrieved it live.

Configure the service worker

The service worker configuration is described in detail in the Angular docs. For caching data, you typically want to configure a dataGroup in your service worker configuration file ngsw-config.json, similar to the following:

 1{
 2    "dataGroups": [
 3        {
 4            "name": "api",
 5            "urls": [
 6                "/api/**"
 7            ],
 8            "cacheConfig": {
 9                "strategy": "freshness",
10                "maxSize": 100,
11                "maxAge": "2d",
12                "timeout": "5s"
13            }
14        }
15    ]
16}
  • This snippet instructs the service working to apply caching to all GET-requests to an address that starts with /api/ (line 6).
  • The freshness strategy (line 9) makes sure that the app will try to get the most up-to-date data from the server but will fall back to the cached data when this fails due to a failing (or absent) network connection. Alternatively, you can use the performance strategy, which will return the cached version immediately, and only fetches new data from the server when the cache has expired.
  • maxSize (line 10) limits the size of the cache to 100 items.
  • The maxAge property (line 11) limits the validity of the data in the cache to 2 days, in this case. After that, the data is no longer considered valid and is removed from the cache.
  • Finally, the timeout property (line 12) is useful in scenarios where you have a bad network: when you have a bad network connection, and you do not get a response in 5 seconds, give up and return the cached data.

Using this configuration, we can fine-tune the behavior of the service worker to suit our needs. We still need one more piece to this puzzle though: something will need to pre-fetch the data beforehand, so that it is available once the app is offline.

Prime the cache

With the above configuration, the service worker is configured to suit our needs. We still need one more piece to this puzzle though: something will need to pre-fetch the data beforehand, so that it is available once the app is offline. We can build a CachePrimer service to do that.

@Injectable({providedIn: 'root'})
export class CachePrimerService {
  constructor(
    private readonly dataService1: DataService1,
    private readonly dataService2: DataService2
  ) {}

  primeCache(): void {
    this.dataService1.getData().subscribe();
    this.dataService2.getData().subscribe();
  }
}

This service has references to all data services that make the API calls to retrieve the data that needs to be cached. When calling primeCache() on this service, it will in turn call the data services to retrieve the data. In the background, the service worker will do the rest. It caches the data and makes it available for use offline.

The only thing left to do is to find a logical place for the call to primeCache(). This could be tied to a UI element, it you want the user to do this as a conscious action, or you could simply call it in the background when loading the app or loading a certain screen. It’s up to you to find the best use of this for your particular application.

Saving data

The second piece of the offline puzzle is to be able to save data. Of course, when you are offline, you cannot save the data back to the server. The basic solution, then, is to create a local ‘queue’ of some sort, which will contain all items that need to be saved. As soon as the app is back online, you process the queue and post each item to the server.

There’s a bit of a catch, however. Remember back in the first part of this series how there might be a bad network connection, but a connection nonetheless? It’s possible that the app is online, but the network connection is just very slow. In that case, we probably still want to try and save our data to the server, but we need to deal with the possibility of timeouts or failed connection attempts.

In short: there’s three scenarios that our app should be able to handle:

  1. Online with a good network connection
    Saving data can work properly using regular POST/PUT requests.
  2. Offline
    The call to save the data needs to be queued, to be processed later.
  3. Online with a bad connection
    In this case, the app thinks that it is online. So, it will try and POST/PUT the data as usual. However, this might cause time-outs, so we need to be able to retry the operation, or simply give up and queue the message.

To implement such a strategy, we need a few ingredients:

  • A SyncTask. This class wraps our network requests. These are the objects that will be put on the queue when the app is offline.
  • A SyncService. This service will contain our queue and handle the logic for sending, re-trying, and queuing outgoing requests.

SyncTask objects represent our API calls

Let’s start with the SyncTask. It’s implementation looks like this:

export class SyncTask<T extends Payload> {
  constructor(
    public url: string,
    public body: T,
    public params?: string) { }
}

Basically, this object wraps the payload that we want to send (in this case, our payload implements the Payload interface). For convenience, we also store the URL and any required HTTP parameters. You can customize this class to fit your requirements. You might want to add a field that stores the HTTP method (POST, PUT, etc.) as well.

A SyncService handles our API calls

The SyncService will contain several elements. First, the SyncService will be our app’s go-to service for sending requests. So, we need some function that will POST data to the API.

@Injectable({ providedIn: 'root' })
export class SyncService {
  constructor(private http: HttpClient) { }

  tryPostPayload<T extends Payload>(url: string, payload: T, params: HttpParams): Observable<T> {
    return this.http.post<T>(url, payload, { params });
  }
}

Retry API calls when the connection is bad

Right now, all this function does is simply try and POST the data as usual and return the resulting Observable. This satisfies our ‘online’ scenario, but will of course fail when offline, or when there is a bad connection. The first thing we can add here, is a retry mechanism. For this example, let’s say we want to have a response from the API within 5 seconds. If the response takes longer than 5 seconds, we considered it a failed request, and try it again 2 more times. We can implement this using the timeout and retry functions from rxjs:

const HTTP_TIMEOUT_IN_MS = 5000;

// ..

tryPostPayload<T extends Payload>(url: string, payload: T, params: HttpParams): Observable<T> {
  return this.http
    .post<T>(url, payload, { params })
    .pipe(
      timeout(HTTP_TIMEOUT_IN_MS),
      retry(2),
      catchError((err: HttpErrorResponse) => this.handleError(err)),
      share()
    );
}

private handleError(err: HttpErrorResponse): Observable<any> {
  // TODO...
}

This is a good start, but we need some additional logic in our error handler to distinguish between different types of errors. First, let’s make sure that we only catch and handle the error when it is a network-related error. We don’t want to catch and handle other server errors here, e.g. authentication errors or other client errors (HTTP 4XX) or server errors (HTTP 5XX). We can do that with this code:

private handleError(err: HttpErrorResponse): Observable<any> {
  if (this.offlineOrBadConnection(err)) {
    // A client-side or network error occurred. Handle it accordingly.

    // TODO: add call to queue to retry at a later time

    return EMPTY;
  } else {
    console.log('A backend error occurred.', err);
    // The backend returned an unsuccessful response code.
    // The response body may contain clues as to what went wrong.
    return throwError(err);
  }
}

private offlineOrBadConnection(err: HttpErrorResponse): boolean {
  return (
    err instanceof TimeoutError ||
    err.error instanceof ErrorEvent ||
    !this.connectionService.isOnLine() // A helper service that delegates to window.navigator.onLine
  );
}

Catching the TimeoutError makes sure that we handle the error that our own timeout function throws. Other network-related errors can be caught by catching the ErrorEvent. And if we have found some other error, but our app knows that is must be offline (because window.navigator.onLine tells us so), then we can also conclude that we need to use our offline error handling.

Queue failed requests to retry later

Now that we can detect when a network related error occurs, we can implement the next part of our strategy: we need to add the failed request to a queue.

const STORAGE_KEY = 'syncTasks';

// ..

private handleError<T>(
  err: HttpErrorResponse, 
  url: string, 
  payload: T, 
  params: HttpParams
): Observable<any> {
  if (this.offlineOrBadConnection(err)) {
    // A client-side or network error occurred. Handle it accordingly.
    this.addOrUpdateSyncTask<T>(url, payload, params);
    return EMPTY;
  } else {
    console.log('A backend error occurred.', err);
    // The backend returned an unsuccessful response code.
    // The response body may contain clues as to what went wrong.
    return throwError(err);
  }
}

private addOrUpdateSyncTask<T>(url: string, payload: T, params: HttpParams): void {
  const tasks = this.getExistingSyncTasks();
  
  const syncTask = new SyncTask(url, payload, params.toString());
  tasks.push(syncTask);
  localStorage.setItem(STORAGE_KEY, JSON.stringify(tasks));
}

private getExistingSyncTasks(): SyncTask[] {
  const serializedTasks = localStorage.getItem(STORAGE_KEY);
  
  return (serializedTasks)
    ? JSON.parse(serializedTasks)
    : [];
}

And with this last addition to our code, we have a working mechanism for queuing our API calls.

Process the queue when we are online

All that is left, is to empty this queue once the app is back online. It is up to you to determine when to trigger this. You could monitor the window.online or window.offline events. Or perhaps you add a Sync-button to the UI to give the user control over the data synchronisation process.

The process for emptying the queue looks like this: you iterate over your queue. Then for each sync task:

  1. Post the sync task to the API.
  2. Remove the sync task from the queue.

Be careful that a simple loop with http.post().subscribe() calls will result in more-or-less simultaneous/parallel calls to your API! If your API can handle the load, and the order of the operations is not important, this may be an efficient way to save your data. But in other situations, you may want to post your sync tasks one-by-one. In that case, you can use code such as the following to make the calls sequentially. The queue is processed FIFO: first-in, first-out.

 1sync(): Observable<any> {
 2  const syncTasks = this.getExistingSyncTasks();
 3  const requests: Observable<any>[] = [];
 4
 5  syncTasks.forEach((task: SyncTask<Payload>) => {
 6    const params = { params: new HttpParams({ fromString: task.params }) };
 7    const obs$ = this.http.post(task.url, task.body, params)
 8      .pipe(map(_ => task));
 9
10    requests.push(obs$);
11  });
12
13  const all$ = concat(...requests).pipe(share());
14
15  all$.subscribe(task => {
16    const index = syncTasks.findIndex(t => t.equals(task));
17    syncTasks.splice(index, 1);
18    this._syncTasks.next(syncTasks);
19    localStorage.setItem(STORAGE_KEY, JSON.stringify(syncTasks));
20  });
21
22  return all$;
23}

In lines 1-10, we build an array of requests, but notice that we don’t call subscribe() on any of the http.post() calls just yet. This builds up our list of the API calls that we need to make to save the data, without actually performing the calls.

Next, we use the concat function from rxjs, to make sure that we can wait for the result of our previous call before we initiate the next call. That is the observable that we subscribe to. Inside the handler, we look up corresponding sync task in our queue, remove it, and save our updated queue.

Hopefully, these recipes and code examples give you some inspiration how you can add offline capabilities to your angular app.

Up next: the final part of this series! In the final part we’ll look at the offline-first scenario: how can you deal with data when you are mostly offline and are only occasionally online?