Keynote Cliff Notes: Red Hat’s Recommendations for UXL Foundation’s Growth
An open source software encourages innovation in collaborative environments. The oneAPI programming framework for multiarchitecture, cross-vendor, accelerated parallel computing is one of the prime examples of an open software paradigm. Intel encourages open source software development and actively contributes to the open source communities such as PyTorch*, Python*, OPEA, etc. It develops its own optimizations of the open source projects, and upstream them to make them available across heterogeneous architectures from diverse vendors for high-performance AI and accelerated computing. Learn about Intel’s involvement in the open software ecosystem – its open platforms and community advocacy contributions.
In the latest oneAPI DevSummit hosted by the Unified Acceleration (UXL) Foundation in association with Intel in October ‘24, Dave Airlie from Red Hat* delivered an insightful keynote session called ‘Open source code is not enough’ where he discussed how an open source project or code alone is not enough to create an open ecosystem such as the oneAPI one. The discussion highlighted the keys to a successful open source development model driven by collaboration of developer communities and enablement of competitors’ platforms, as opposed to just focusing on the open source code. Open source projects such as the UXL Foundation’s oneAPI projects were the session's focus.
This blog will give you an overview of the keynote.
About The Speaker
Dave Airlie, a distinguished engineer from Red Hat, is one of the major contributors to the Linux GPU stack across the kernel and user space. He works on Linux and AI products in the GPU and accelerator space at Red Hat.
|
The Open Source Concept: An Overview
There are several aspects of an open source project, as follows:
- Code,
- Release method,
- Development method, and
- Continuous integration and continuous delivery (CI/CD)
From [00:03:10] in the session recording, the presenter discusses the following categorization of origins of open source projects:
- Single developer bootstrap – Individual developers or folks from a company bootstrap a project independently or as a small group of innovators. The community gradually grows, and the project develops within the community.
- Corporate product opening – A corporation develops a project internally and later decides to open source it for business reasons, for example.
The UXL Foundation’s oneAPI projects fall under the category of corporate product opening.
Open Source Corporate Product Development
The internal process of corporate product development comprises 3 major steps, as discussed in the session from [00:04:36]:
- Stage 0: Over-the-wall release method – The very first step in corporate product development is opening the code or making it available externally, which can be done in several ways. Some project owners prefer releasing only certain parts of the project that they want to open source, while others release the whole project simultaneously.
- Stage 1: Creating a community project – In order to make the project accessible to the external communities, some aspects of the project, such as the development stage and the release stage, are performed outside of the corporate wall. The company does some public planning through platforms like GitHub*. Whereas a lot of the planning process, the CI/CD infrastructure, etc., still remains internal to the company. There are reasons behind it such as security concerns posed while creating external CI/CDs.
- Stage 2: Adding companies – Suppose a new company joins the project. It may have its own project planning and engineering teams. It may also have conflicts with other companies associated with the project, making it difficult to manage project planning and communication among various companies. A solution to such an issue is to have the project at the center and allow all the companies to access it and work on their version independently.
Dave also talked about a beneficial method of categorizing products and projects. The product and the project may have different names, goals, project maintainers, and developer teams. An example of this is the oneAPI product and the oneAPI project. However, it is important to make a call, whether a project should have an associated product (a process called ‘commoditization of the project’). For example, Linux kernels are all commoditized, where the projects are built downstream (based on the results from the previous version(s)), and the distributions are usually the products.
The session also discussed the types of developers building community projects. Below are the three categories:
- Trailblazer – the one who tackles hardware and software requirements for achieving a specific goal say, running a oneAPI project on some new hardware.
- Road builder – the one who builds the project and the product out of it step-by-step.
- Long-term maintainer – those are the people who take care of the project for the long term. Several large companies either have a separate maintainer role or a senior engineer who handles tasks like cleaning up the code; while for open source projects, such developers focus more on delivering new features than maintainability.
A project needs all three types of developers mentioned above. However, not all of them are required at all project phases, and one may switch roles.
As discussed by Dave, project maintainers should:
|
From [00:20:33] in the session recording, Dave discusses what makes a successful open source project:
- Multiple companies should collaboratively develop the project. While some parts of the project may be specific to certain companies, all the companies should play their roles in developing the core of the project.
- Multiple maintainers from diverse companies should make decisions based on the long-term interest of the project instead of the short-term interest of the product.
- Different types of developers (trailblazers, road builders, and maintainers) should coordinate without conflicts.
- Resilience is an important factor in the project development process. If one or more participating companies decide to drop in between, the project should still be able to continue well.
From [00:23:20] in the session recording, Dave also discusses how we get the oneAPI Ecosystem that the UXL Foundation is trying to build, keeping the above factors in mind.
- Investing in the common core: All the participating companies should invest in developing the open-source project's common core instead of prioritizing only their own products.
- Enabling competitors’ devices: The UXL projects should be compatible with competitors’ hardware, enabling support for multi-vendor architectures and attaining a competitive position in the market.
- Balancing efforts between developing new products and maintaining existing project components.
Intel Embraces an Open Software Ecosystem
We encourage open source software development and innovation through our active contributions to the UXL Foundation’s oneAPI initiative of multiarchitecture, cross-vendor, accelerated parallel computing. We also upstream our optimizations to open source projects including OPEA, PyTorch and TensorFlow so that developers can avail themselves of the latest and performance-enhancing features with their existing codebase.
We drive sincere efforts to create an open AI ecosystem in collaboration with key industry players including large-scale companies, startups and academic institutions. For instance, we have joined hands with Red Hat to deliver open source AI workload potability across hybrid cloud infrastructures. The Red Hat OpenShift* platform integrates the complete feature-rich and optimized AI software portfolio, and enables application deployment across the latest Intel architectures including CPUs, GPUs, and Intel® Gaudi® AI accelerator.
What’s Next?
Check out the complete session ‘Open source code is not enough’ held at the oneAPI DevSummit. Other interesting talks and discussions from the event are available here.
We encourage you to explore our oneAPI-powered libraries, tools, and framework optimizations for high-performance AI and HPC development.
Additional Resources
Technical Software Product Marketing Engineer, Intel