Skip to main content

Steps to Implement Datafree Reach

  • Updated
 

Introduction

Converting a Website to run datafree within a browser requires every component that forms the website to be pulled from special IP addresses that have been registered with all the Telcos to be Reverse Billed. For most sites this a straightforward process taking a few hours and requiring little or no changes to the website code. For more complex and high volume sites there may be some more effort and it may require some modification to the website components. 
 
This document outlines steps that can be followed to make a website datafree. For most implementations only a few of these steps will actually require changes to be made. However, you should review each step and verify whether it is applicable to you.  
 

1. Review and Scope

Task

Review the website and consider its structure. A good starting point is to load the website in a browser and review the components that are included with it.  Datafree provides the Datafree Visualiser tool to help you to understand the traffic that will need to be managed to make the site #datafree. The developer tools provided in most browsers also help.
 
It is worth looking at the data size of the components on the site and deciding early whether some can be shrunk or optimised early in the conversion process. Early decisions can generate big saving in your data cost.    
 
You may not want to make the whole website datafree. Identifying early any sections that you do not want to cover can make the conversion process a lot easier.  
 

Technical Background

  1. Some website development frameworks and responsive design solutions can add a large data cost. Decide if they are required.  
  2. For most online advertising the data cost is often greater than the revenue earnt. Some advertising platforms cannot work datafree.
  3. Image heavy and video content can be a large proportion of cost. Images typically use 10x the data of text and videos are 100x text for similar minutes of user engagement. Limiting these can have a huge impact on data costs.
Tips
  1. Use Datafree's Datafree Visualiser to show the actual requests make when navigating the site
  2. Use the developer tools within most desktop browsers.
  3. You can connect a PC to your mobile and use the developer tool in a desktop Chrome browser to report on your website being loaded in a mobile browser.   

2. Build a First Cut datafree website 

Task

Datafree provides the Datafree Workbench to help you make your site datafree. It can automate a lot of the work required to make it happen. The process records all traffic while you navigate the site and then analyses the traffic to automatically build a configuration to Reverse Bill the site.
 
This will usually convert most of your website to datafree in a few minutes. If there are areas that require further attention, it  is a valuable first step to identify anything that needs to be addressed. 
 

Technical Background

The Datafree Reach product directs all the network traffic for your site to pass through a Datafree Gateway by changing the domain name for all the components in the website. It does this by discovering all the URLs in the site and then generating special rules to modify them ‘on the fly’ as they move from your webserver to the browser. It can take a few iterations to find all the scenarios and develop the right rules.
Tips
  1. Register an account at the Datafree Portal and set up your ‘Reach’ project. 
    1. Login the the Datafree Portal: https://pub.bi.nu
    2. Register an application of type Reach
    3. Register your own unique subdomain under datafree.co that suits your business
  2. From there you will find a link to the Datafree Workbench. The Workbench has detailed instructions on how to use it. 
  3. Create a ‘Dev’ version of your App within the Datafree Portal. You can develop your datafree configuration by running against your production website. 
  4. The ‘Dev’ version is a sandbox environment where you can develop rules and run against different versions of your website. When you are ready you can migrate the rules to a ‘Production’ version of the app.  
  5. A lot of your site can be reverse billed automatically by the tools in the workbench. Where this does not happen, try to find temporary workarounds for any issue so you can keep exploring other areas of the site.. For example, if the site login does not work, can you disable it in your test system so you can still access the whole site? 
 

3. Refine Configuration

Task 

The First Cut datafree website that is generated within the Datafree Workbench may be a complete solution that you can go live with. However, there may be some refinements required to improve performance, increase reliability, save data or to remove elements that cannot be made datafree. 
 
The changes may involve refinement of the configuration with the Datafree Workbench but some changes may need to be made to the website itself. If required, for most websites the changes are usually quite straightforward and can be completed with a few hours effort. 
 

Technical Background

Some of the changes that should be considered are covered in more detail in the next steps. 
Tips
The following items are some of the issues that may need small refinement within the Workbench: 
  1. The auto generation process generates new ‘free’ domain names that point to the Datafree Gateway. Sometimes the defaults are too long and need to be manually shortened in a way that ensures that they are unique and still meaningful to you. 
  2. If you have domain names that contain variable values (common with content distribution networks) each variation will be seen as different domains and duplicating rules will be created. Change these to use wildcard rules to simplify your configuration. 
  3. Using Relative URLs in your content reduces the amount of processing required to convert domain names and can speed website delivery. Change your website to use relative URLs where possible. 
  4. The inclusion of some content in your website is protected with extra security such as CORS. The rules or implementation may need to be refined, see here for more details.  

4. Identify Services that can’t be Reverse Billed. 

Task

There are a number of popular services used in websites that for technically and/or legal reasons cannot be reverse billed within a browser. Often the restrictions are architected into such products for reasons of security or control. 
 
It is important to identify these and determine how to address the limitation. 
 
Many services are identified early when you build your First Cut datafree website. Reviewing the Reach documentation to identify any others.  
 

Technical Background

Restricted services include: 
  1. Most third party login services such as Facebook or Google login. 
  2. Most Ad Retargeting products. These use cookies that cannot be set within the browser in a manner that such products require. 
  3. Some Google Firebase services. See here for more details.
  4. Some services like Google Analytics can be zero rated but there are implications that may be significant. See here for more details. 
  5. Some third party services have Firewalls and Denial of Service protections that can be upset by channelling requests through the Datafree Datafree Gateway resulting in blocked requests. 
  6. Google Maps has contractual limitations and these are enforced with security checks. 
  7. Youtube videos cannot be zero rated. It is recommended not to use youtube anyway as the videos are served with a large data overhead of advertising and tracking controls - even if these are not used. 
  8. Some third party services may not correctly respect the x-forwarded-for header in the HTTP protocol. This can cause side-effects, e.g. users can be geolocated incorrectly.  
Tips
Look at the different components and functionality in your website and determine how they are delivered. Services / Features to consider include: 
  1. Layout frameworks
  2. Content Distribution Networks (CDN)
  3. Login Services (Google Facebook)
  4. Advertising
  5. Advertising retargeting
  6. Analytics
  7. Websockets eg Chat Sessions
  8. Centralised Data Storage (eg Firebase)
  9. Youtube, Google Maps
  10. Notifications (GCM, FCM)
You can use the Datafree Visualiser to see all the domains that content is drawn from. This can help to identify services that need to be considered. It is important to work through each source and understand what it is for, whether it is needed and whether work is required to zero rate it. 
 

5. Identify Data Heavy Services

Task

Many organisations think they have data optimised their sites but when they start paying for the data they quickly realise that there is a lot of fat that can be trimmed. 
 
The earlier you start to think about data savings the better. 
 
It is usually more effective to spend your reverse bill budget on increased user engagement than on heavy graphics. 
 

Technical Background

Many components of the website are very inefficient in their use of bandwidth. Eliminating these can save you 80% of your reverse billing costs. 
  1. For most online advertising the data cost is often greater than the revenue earnt. 
  2. Image heavy and Video content can be a large proportion of cost. Images typically use 10x the data of text and videos are 100x text for similar minutes of user engagement. Limiting these can have a huge impact on data costs.
Tips
Use the Datafree Visualiser to identify the large components of your website. Typically 80% of the data is used to deliver 10% of the content experience. Try to optimise these elements, decrease the quality of the images, do you really need 2 million colours in an image? If you cannot optimise, consider removing the data heavy sections of the website when the user is reverse billing. Focus on delivering the functionality that delivers the greatest benefit.
 
Caching can have a huge impact on data usage (and website performance). Review the caching settings for all the components in your website so that stable content stays in the browser cache and is not downloaded unnecessarily.
 
Ensure that all resources such as javascript and font files are really needed.
 

6. Login Systems

Task

A number of the integrated login services provided by Facebook, Google etc cannot be zero rated. For their own security reasons these companies explicitly block the logins when the traffic is being forced via a Proxy/Gateway like Datafree's.
 

Technical Background

To enable Reverse Billing of your website, Datafree uses a ‘two certificate’ security system. The transfer of passwords with this approach does not satisfy the security standards that these services enforce so they cannot be used. In addition there are technical issues with mapping of domain names, cookies etc that prevent them from working as intended. 
Tips
You have 2 options: 
  1. Use your own login system. Most organisations have this as an alternative to the Google/Facebook login systems so it may just require the removal of the third party services. 
  2. Continue to use the services and accept that it will use a small amount of user’s data. If you do this your site will not work when the user does not have data, nor will you qualify to promote your site as a datafree site. 
See here for more details.
 

Task 

Many websites store data about a user or session in Cookies. Cookies are associated with a domain name and this association determines when they are used and their ‘scope’ for sharing. With the manipulation of the domain name to be free domains the ‘scope’ of a cookie is affected. The datafree gateway alters the association of cookies to domains to ensure they are stored and work. However, this may impact their ‘scope’ and possible implications should be checked.   
 

Technical Background

The Datafree solution for Reverse Billing depends on rewriting the domain names of URLs. The system of cookies within a Browser works to store data in the browser by associating the data with domain names. When the domain name is being manipulated the association between the stored data and the domain name must match for the data to be stored. If required, the Datafree Gateway automatically modifies the scope to ensure there is a valid association.
 
Due to issues with HTTPS certificates there are limitations on the structure of the ‘free domains’ used in Reach so that cookies defined for paid domains have to be carefully handled by the optimiser to work with free URLs. Ideally cookies should be linked to the full domain name in the URL (in the set_cookie instruction no ‘HOST’ parameter should be supplied). If the cookie must be set at a higher level domain, the cookie must be small and it should be done sparingly. The cookies must not contain any secure or private data as their security may be compromised when they are linked to the free domain
  
Cookies stored against a free domain are not available if the same user visits the website directly (i.e. using the site’s own ‘paid domain’) and vice versa. 
Tips
  1. Check the existence and scope of cookies via the Datafree Workbench. 
  2. When the ‘scope’ of a cookie is restricted by the path in the URL, this scope is not changed.  
  3. There are techniques that can help keep cookies in sync between ‘paid’ and ‘free’ domains. Contact Datafree support for more details. 
 

8. Test and Verify. 

Task

Once your website is working in a single browser, make sure you test in the different scenarios in which it is run.
Ongoing, remember to retest after any changes are made to your site. 
 

Technical Background

Websites are complex and dynamic and a number of special cases may not have occurred when you created your .Har file or did your basic testing. 
Tips
These are some things to test for:
  1. Test the datafree website on a mobile (especially if your website uses a different subdomain (eg m.companyname.com) for mobile users. Ideally use a phone with a SIM card with a zero data balance so you experience the site like your users will. 
  2. Start the website using an http:// url rather than https://. Many websites redirect to the https:// url to ensure all traffic is secure. This may cause problems.  
  3. Do users access the website via other domain names that redirect to the main domain name. These may be domains from legacy systems. Do you need to add these to your list of mapped domains? 
  4. Does your site have dynamic content that may occasionally pull in content from other sources. Those sources may not have been in your .Har file so no domain rules created for it.  
  5. The traffic to your backend will all appear to be coming from a single IP address (our server) rather than from the IP address of the user’s phones. If your system uses an IP address to identify or locate users you may need to make changes. The user’s IP address is available in the x-forwarded-for header and in a special ‘Datafree’ header. 
  6. Does your backend (firewall etc) limit connections from a single IP Address. Such security rules may be triggered when all the traffic is coming via a single IP address (our server’s). 
 

9. Make datafree visible. 

Task

To maximise the benefit of datafree it is important that your users know when it is datafree. It is also important that to minimise and confusion or doubt about any limitations of datafree. Building the user’s trust will allow them to explore more and engage more.    
 

Technical Background

To a user a datafree website looks very similar to a traditional site where they would be paying for their data. The only difference may be the ‘datafree’ in the site’s URL. Therefore, Datafree offers tools to help you inform your user about datafree and show them when they are, and when they are not enjoying the benefit. 
 
Within a browser, it is not possible to determine with a 100% accuracy when a user is enjoying datafree; when they are using WiFi; or they are paying for their mobile data (e.g. there is no datafree agreement with their Telco). The Datafree system uses tools to try to determine this and it can indicate to the user when the connection appears to be datafree.   
Tips
  1. When promoting your website as datafree, explain how it works and how they can spot a datafree website.  
  2. The Datafree Gateway can inject a special datafree icon into the website’s pages. Turn on this feature for your ‘Free’ zone and the icon will ‘floats’ over all pages. 
  3. Display a welcome page when the user arrives at your datafree website. The standard datafree welcome page explains datafree to the user before they enter the site. 
  4. Develop a bespoke ‘Welcome page’ for your website which conveys your own message.   
  5. Manage links that leave your datafree pages and will cost the user their own mobile data. It is important that a user knows the limit of the content you make datafree. When a user clicks on links to other websites (or content on your own side that is not being offered datafree) a warning page will be displayed to the user advising that they are leaving the datafree site and giving them an option to continue using their paid data or return to your datafree pages. A standard warning page will be displayed but you can develop your own page with your own message. 
 

10. Driving users to your Reverse Billed Website. 

Task

You need to consider how users will get to your website. 
 
Most users do not remember URLs, nor do they use Bookmarks within Browsers. They navigate to sites every time via a search engine. However, those search engines will not be available to users who have no data. If you want to reach users who have no data, you need to plan this in to your strategy. 
 

Technical Background

Users with no data cannot use a search engine to find your website every time they want to find it. You need to make it easy for them to get to it and remember where it is. 
Tips
  1. Use the Datafree URL Shortener service so you have a very short start URL. 
  2. Develop your website as a Progressive Web App so that it appears as an icon on the user’s home page. 
  3. Encourage your user’s to bookmark the website. 
  4. Many users have data bundles for Whatsapp or Facebook, but no data to reach your site or use a search engine. Consider using these platforms as the path by which users navigate to your website. 
  5. If a user has no data then they cannot communicate with any part of your website that is not reverse billed. Therefore you cannot drive users to your main website and  then ‘redirect’ them to the reverse billed site when they are on mobile. A better strategy is to drive all users to your reverse billed site then bounce those not using mobile devices to your normal website. 
 

Related articles