From Burning Oil Platforms to Web Platforms: Meet Riju Bhaumik

Like a phoenix rising from the fire of a burning oil rig, Riju Bhaumik shares his journey as a web platform engineer at Intel.

Starting as a developer, then a committer, he’s currently the code-owner of three modules in Chrome* upstream: ImageCapture, WebNFC and Sensors. He's also an editor of a couple of specs in Devices And Sensors (DAS) Working Group and on the WebRTC Working Group in the World Wide Web Consortium (W3C).

Q: You were at the 2022 Intel Innovation conference showing the latest code and demos that will change the way video conferencing on the web works. What kind of feedback did you get? 

A: A subset of the audience was wowed, but most were surprised to learn that Intel was working on “browser-code” and in some cases leading the work. Even some Intel colleagues were mildly astonished that our company has been a significant contributor to the web platform for over a decade. While it’s more commonly accepted that Intel is the top contributor to the Linux* kernel and active on other open-source projects like Kubernetes*, TensorFlow*, PyTorch*, LLVM*, etc., where it’s expected for new hardware enablement and artificial intelligence accelerators, Chrome seems too far up the stack for a company like Intel.

Q: Rolling back to the start of your time at Intel, when Stephen Elop talked of burning platforms to motivate Nokia to address the rise of Apple and Android phones in February 2011, it sealed the fate of MeeGo*.  Intel was lucky enough to recruit many engineers who worked on MeeGo, like you.  Can you tell me what it was like to join Intel Finland? 

A: The Software and Services Group (SSG) I got hired into at Intel was like the biggest software company that no one had ever heard of. Our team started working on the WebKit* project which, in addition to powering Safari*, was ported to several toolkits like GTK*, Clutter*, Qt* and provided the web engine for most consumer products at the time. Intel quickly became the third- most influential contributor to the WebKit project.

Q: After MeeGo, Intel worked with Samsung* to try to create an application ecosystem for HTML5 mobile apps for Tizen* devices. How did WebKit meet that challenge? 

A: We worked with Samsung to create a Tizen Web Runtime based on WebKitEFL.  When Google forked WebKit’s WebCore to create Blink*, our team transitioned to the Chromium* ecosystem. We started a new Web Runtime called Crosswalk on top of Chromium, for cross-platform mobile development targeting ambitious (HTML5) web apps and based on the now-defunct System and Applications Working Group, better known as SysApps API.

Q: At the time, people were used to the web being a series of websites delivering relatively static content to a web browser. How did the rise of HTML5 change that perception? 

A: The idea of the web as a platform for applications and not just a document reader had been on the minds of many, giving rise to various attempts to package the web platform as a standalone runtime. Adobe AIR*, Chrome Packaged Apps, Firefox* OS packaged apps, Apache Cordova*/PhoneGap*, Electron*, etc. all grew in popularity. Meanwhile, the Chrome team at Google* had been adding new exotic features to deliver experiences not seen on the web before. They set up transparent processes to implement and launch W3C standards-based features.

This culminated in the creation of Progressive Web Apps (PWAs), now a widespread name, created by Alex Russell and Francis Berriman with the help of a large community, including our very own Kenneth Rohde Christiansen.

Q: Did PWAs provide the same capabilities as native applications? 

A: We took learnings from SysApps and Crosswalk and collaborated with Google, Microsoft*, Samsung and others to create the “Capabilities Project,” better known as Project Fugu, to bridge the gap between native and web applications and empower web applications to push the boundaries of what’s possible in the browser.

Q: Let’s address the elephant in the room: “All this is nice and good open source citizenship, but how does it make business sense for Intel?”

A: Our internal research shows that on Intel client devices, users spend more than 60% of their time on the web platform. Productivity apps from Microsoft, like Office 365*, are now web based. Web Assembly (WASM) along with various Fugu APIs like File Access have enabled “demanding” applications like Photoshop* and Autocad* to thrive on the web.

The pandemic has also accelerated the adoption of online collaboration tools. Google Meet* is Web based and Microsoft Teams* and Zoom* have significantly invested in their web apps, all powered by WebRTC (Web Real Time Communication), WebCodecs and WebTransport.

Q: Obviously we want to ensure that this major piece of the PC ecosystem provides users a great experience. How do we do that? 

A: The web platform provides companies like Intel an open ecosystem to influence implementation of features on Chrome/Edge (~9,000 patches upstream) using standards-based APIs ratified by the W3C. Intel engineers chair 4 W3C Working Groups and many more are editors or provide input to multiple specifications. Intel is moving where the end users are spending their time to deliver enhanced experiences. Intel is collaborating with the main stakeholders of the web platform to bubble up the hardware goodies on the chip and translate them to experiences all the way up to the users.  It’s no longer sufficient to just do the platform enabling device drivers. It’s of paramount importance that we embrace a “virtual vertical” approach with our partners across all levels of the stack, from hardware co-design to end-user experience, because users and developers, our main stakeholders, care about experiences and ecosystem. Working on Chromium upstream and W3C standards maximizes our reach not only to Chrome’s 3.3 billion users, but also to hundreds of downstream projects like Microsoft Edge*, Opera*, Brave*, Vivaldi*, Yandex*, Electron*, etc. and to the highest number of developers.

Q: You’ve been exposing hardware sensors to the web framework.  Can you give us some examples of the work you’ve been enabling? 

A: Intel Sensor Hub (ISH) is a key part of Intel IP to offload sensor polling to a low power co-processor from Kaby Lake onwards. I started first by exposing Ambient Light Sensor events on Chrome browser. Soon the team realized the status quo could be improved upon, which led to the Generic Sensors project to encompass all sensors on a client. Today, sensor data is used in many platform-specific applications to enable use cases such as immersive gaming, fitness tracking and augmented or virtual reality. Using Generic Sensor API, developers can feature-detect and configure different sensors for their varied use cases.

I started to expose NFC functionality to Chrome browser when Intel was serious about Near Field Communication (NFC). Intel bought Inside Secure*’s NFC business, upgraded its membership of NFC Forum, had a seat on its board of directors and partnered with MasterCard* and Microsoft to bring NFC to laptops (also Intel powered smartphones like Lenovo K800 and Medfield based Android tablets) to work with Intel’s Identity Protection Technology.

To support Intel’s IOT and Augmented Reality projects, I maintained the (now removed) Intel RealSense™ Technology code path on Chrome upstream.  That work led to authoring a new WebGL extension (originally used for RealSense* optimization) and driving it to standardization in Khronos. It's now implemented in Google Chrome, where Unity and other medical imaging companies are major consumers of this work.

Q: You’re now ‘focusing’ (pardon the pun) on cameras and AI.  What capabilities are you adding there? 

A: Cameras and AI are now a big part of laptops since we depend on video conferencing a lot these days. Unsurprisingly, Intel has multiple teams working on camera, IPU (Image Processing Unit) and client AI solutions. MediaStream Image Capture is the specification to control the camera's properties like focus, exposure time, brightness, color balance, pan-tilt-zoom (PTZ), etc. from a browsing context. Intel’s Client Computing Group (CCG)’s Intelligent Collaboration experiences on our new Evo* devices are exactly what we need to enthrall our users with. We’ve been working with the wider web community (Meet, Teams, Zoom, etc.) to bring background blur, face detection, eye contact correction, automatic framing, and many more to the web platform.

We also worked with the WebCodecs team to bring hardware acceleration for media processing to the Web. Our efforts on VA-API helped many downstream customer projects like the custom-built Chromium on Kylin* OS.

Q: How has the open source community reacted to your contributions? 

A: No progress happens in a vacuum. All the efforts of my team members (present and ex) and a lot of advice from our Chrome friends helped us earn the trust from the open source community. Sensors, NFC and mediacapture-image are modules on upstream Chrome where I have code-owner responsibility (like the maintainer role in the Linux* community). Our work was highlighted at various conferences like BlinkON* and Chrome Dev Summit and my hobby project that has earned 235 GitHub* stars.

Q: What specific areas of collaboration are you looking at for the future? 

A: Our web team at Intel works across all domains of the web platform - Fugu, Media, WebRTC, WebGPU, Web Assembly, WebNN and so much more. If you want to bring your innovations on the web platform, please reach out to me on Twitter*!

About the author 

Chris Norman is an Open Source Advocate who has promoted the use of open source ecosystems for over a decade.  You can find him as pixelgeek on Twitter,  Mastodon, IRC and GitHub.