OS Development

[ibtjc4everaol_com]

Hi,

I'm interested in developing my own operating system (for now, just as a personal project) for the purpose of gaining a better understanding of all that's involved in OS design and development. Because of not knowing yet if this would be something I would like to take seriously and pursue as a full time project, my interest is not about making money (at least not at the moment).

The problem is that there doesn't seem to be ANY guidance (absolutelyNADA!)from Intel about developing OSes with their own technologies (Amazing!?! Isn't Intel interested in selling microchips now and in the future?).

Anyway, after downloading and reading all five of Intel's 64 and IA-32 manuals a few times over, there's still a problem... BIOS and hardware.Unfortunately simply reading the manuals is not enough for you to go out and buy the hardware you need and begin developing your own OS.You need the involvement of both BIOS third parties and hardware manufaturers. I've discovered that the BIOS is "tuned" to specific OS platforms and the same is the case for hardware as well. Being that I'm not a part of industry, this is something that I don't understand. As you search the market, you will find that, for example, a hard drive will specify what platform it's designed for, i.e. Windows or Macintosh or Linux. Same goes for motherboards with the BIOS.

So is there anyone from Intel out there that can give me SPECIFIC guidance as to step one, step two, etc., as to what I need to know and do to begin my OS development endeavors.

I'm not interested in using Linux or any other kernerl or derivative or pre-packaged OS that I can take and make my own. I want to do like Microsoft, like Linux, like others (that are not some flavor of Linux), I want something from scratch, that has different purposes and goals in mind. Not necessarily being a general purpose OS. My goal is to take advantage of the hardware as I see fit, in my own way, for my own particular purposes.

I'm not a company, I'm not interested in starting a company, not interested in challenging Microsoft or Apple or Linux, I just want to do my own.

To me it seems that Intel's position is to only serve those who are in it for a profit. It seems like Intel discourages those like me, the enthusiast who just is fascinated with technology and want to take it and do things with it in unconventional ways, that a lot of times end upspawning new platforms and creations that no one ever envisioned.

Thanks.

Kevin Gregory Smith - Terrytown, La.

P.S.

Anyone who is not familiar with OS development and who will only create a post to discourage from creating my own OS, do us all the favor and don't bother posting any of your "lack of innovation, imagination and exploration" spirited comments. An Extra Thanks To You All :)!

[ibtjc4everaol_com]

Quoting Adam Kachwalla

Now I am not here to discourage you from creating your own OS (which has happened to me too; I am developing my own OS) but perhaps the reason Intel doesn't provide any sort of guidance is because it is not like every OS is the same. Mine will be different to yours, which will be different to an OS built by another person.

My advice would be to modify some existing OS and slowly introduce the architecture you are after. Perhaps an open source version. Something like ReactOS since it is under GNU Public license.

And it is not like Intel are discouraging you; in fact there is the System Architecture model for many chips. It is just that creating an OS is rather time consuming. And don't expect to finish an OS that is developed entirely from scratch either in a small amount of time. Although you can finish it, it will take a very long time.

Hi Adam,

Thanks for your suggestions, but I think I need to clarify something. I'm not looking for or asking Intel to develop my OS for me. When I state that I need guidance on OS development, I'm referring to their 64 and IA-32 architecture facilities and the "proper" and "efficient" way to utilized them. I'm not looking for training on schedulers or memory managers and things like that, those things are my responsibility and are abstractions. I need guidance with the foundation upon which those abstractions are built. And only Intel would know better than anyone else on the planet as how to best use those facilities.

As far as modifying an existing OS, once again as I stated, I don't want to do that. No GNU stuff, no public license stuff, my own. Just like others who develop the open source tools that everyone uses, I want to develop my own. Whyare the open source people the only ones who get develop tools and gain a better understanding of how the system works? Everyone always suggests, use this already available tool, or this already available tool. Just like they create the linkers and loaders and assemblers that everyone uses, I want to know what they know. Understand? Why should I be kept from the deep knowledge and understanding that others have?That's the whole purpose of my endeavor. I totally understand that this is not something that's going to take a few months to accomplish.

Thanks.

Kevin Gregory Smith - Terrytown, La.

[David_S_Intel3]

Hi Kevin,

Let me respond regarding the Itanium architecture. All of the Itanium processor architecture and platform firmware interfaces that were used for open source Linux development are available publicly. The documents you would need to reference are:

Intel Itanium Architecture Software Developer's Manuals
http://www.intel.com/design/itanium/manuals/iiasdmanual.htm

Intel Itanium Processor Family System Abstraction Layer Specificationhttp://download.intel.com/design/itanium/specupdt/245359.pdf

Unified EFI Specification
http://www.uefi.org/specs/

Intel Itanium Processor Family Error Handling Guide
http://download.intel.com/design/Itanium/Downloads/24927802.pdf

IA-64 Software Conventions and Runtime Architecture Guide
http://download.intel.com/design/Itanium/Downloads/245358.pdf

OS Machine Check Recovery on Itanium-Based Systems
http://download.intel.com/design/itanium/320482.pdf

On Itanium architecture-based platforms, the System Abstraction Layer (SAL) and Unified Extensible Firmware Interface (UEFI) provide all of the platform services you need for OS boot and runtime services.

Regarding I/O devices, you'll need to work directly with the device manufacturer to find out what information they publicly disclose for open source OS driver development.

Intel provides OS developers with the architectural specifications needed to develop operating systems for Itanium architecture-based platforms. However, we don't have step-by-step guidance on how to develop an operating system. There are many design trade-offs to consider, and we let OS developers use their domain expertise toprovide value and innovate.

If you want some insight into the Linux design considerations for the Itanium architecture, you may want to refer to the HP Press book titled "IA-64 Linux Kernel: Design and Implementation" by David Mosberger.

David

[ibtjc4everaol_com]

Quoting David Song (Intel)

Hi Kevin,

Let me respond regarding the Itanium architecture. All of the Itanium processor architecture and platform firmware interfaces that were used for open source Linux development are available publicly. The documents you would need to reference are:

Intel Itanium Architecture Software Developer's Manuals
http://www.intel.com/design/itanium/manuals/iiasdmanual.htm

Intel Itanium Processor Family System Abstraction Layer Specificationhttp://download.intel.com/design/itanium/specupdt/245359.pdf

Unified EFI Specification
http://www.uefi.org/specs/

Intel Itanium Processor Family Error Handling Guide
http://download.intel.com/design/Itanium/Downloads/24927802.pdf

IA-64 Software Conventions and Runtime Architecture Guide
http://download.intel.com/design/Itanium/Downloads/245358.pdf

OS Machine Check Recovery on Itanium-Based Systems
http://download.intel.com/design/itanium/320482.pdf

On Itanium architecture-based platforms, the System Abstraction Layer (SAL) and Unified Extensible Firmware Interface (UEFI) provide all of the platform services you need for OS boot and runtime services.

Regarding I/O devices, you'll need to work directly with the device manufacturer to find out what information they publicly disclose for open source OS driver development.

Intel provides OS developers with the architectural specifications needed to develop operating systems for Itanium architecture-based platforms. However, we don't have step-by-step guidance on how to develop an operating system. There are many design trade-offs to consider, and we let OS developers use their domain expertise toprovide value and innovate.

If you want some insight into the Linux design considerations for the Itanium architecture, you may want to refer to the HP Press book titled "IA-64 Linux Kernel: Design and Implementation" by David Mosberger.

David

Hi David,

When I wrote that I was looking for guidance with the Intel 64 and IA-32 architecture, I wasn't referring to the Itanium architecture. Maybe there was a misunderstanding the way I wrote it, but I'm referring to the x86 architecture for 32 and 64 bit which includes the Pentiums, Xeons and Cores i.e. Duos, Solos, i7s, i5s, etc. Also, I'm not looking for Intel to teach me how to write an operating system (schedulers, memory managers, etc.) I'mneeding guidance from Intel on how to best utilize the facilities that are availabe on it's x86 architecture chips, how to set up paging, task switching, etc. I'm looking for assistance for the topics covered in the manuals that can be found via the following link: http://www.intel.com/products/processor/manuals/index.htm.In addition to where do the BIOS companies fit in and do I also have to work directly with them as I would have to do with the I/O device manufacturers.

Thanks,

Kevin Gregory Smith - Terrytown, La.

[SHIH_K_Intel]

To say that the software developer's manual set is not sufficient to develop an OS, you are absolutely right. I can also say unequivocally that these manual set will not be sufficient to develop an application, either. Both are outside the scope of these manuals.

I personally believe that before starting a software project (whether it's an OS or an application), one must have well defined set of requirements, feature set support, UI design, architecture work, before choosing whether to implement for a particular hardware vendor or a particular generation of HW. Those disciplines are all outside the scope of the manual set.

Sorry I can't give you the answers you like to hear. I also think learning the experience that other people or projects had gone through areextremely valuable resources to bypass. Particularly on the importanceof avoiding feature creep and limiting the project scope.

[ibtjc4everaol_com]

Quoting Shih Kuo (Intel)

To say that the software developer's manual set is not sufficient to develop an OS, you are absolutely right. I can also say unequivocally that these manual set will not be sufficient to develop an application, either. Both are outside the scope of these manuals.

I personally believe that before starting a software project (whether it's an OS or an application), one must have well defined set of requirements, feature set support, UI design, architecture work, before choosing whether to implement for a particular hardware vendor or a particular generation of HW. Those disciplines are all outside the scope of the manual set.

Sorry I can't give you the answers you like to hear. I also think learning the experience that other people or projects had gone through areextremely valuable resources to bypass. Particularly on the importanceof avoiding feature creep and limiting the project scope.

Hi Shih,

I guess I'm just not being as clear as I thought I was. It seems that no one is able to understand what I'm asking. I agree with you; developing an OS or an application is outside of the scope of the manuals and the manuals are not sufficient to develop an OS... that's not what I'm looking for.

When I say OS development using the x86 architecture, I'm referring to the way the facilities of the x86 architecture are to be utilized. I'm not looking for Intel to teach me OS theory and about modern OSes. I guess the best way to explain is with an example.

One of the manuals discuss 'Protected-Mode Memory Management', 'Paging', 'Interrupt and Exception Handling', 'Task Management', etc. Now, the documentation speaks about the data structures related to each of these facilities and how to properly set them up among other things.This is what I'm referring to when I mention OS development. Isn't the OS implemented using these facilities? I want to know does Intel offer guidance (besides the documentation), support, etc., for using these facilities properly and efficiently. If I want to execute some piece of code, certain data structures have to be properly initialized before doing so. That's what the documentation explains and the documentation as a whole is what I'm referring to about OS development. Though the documentation explains quite a bit, there are still questions I have, and many topics that are related aren't presented together which causes confusion as to which "pieces of the puzzle" one needs to properly execute certain code.

The OS itself and everything in between are all abstractions. But when these abstractions need to execute, the facilities of the x86 architecture is how things are get done. No matter how sophisticated an OS may be, when it all boils down to it, these data structures need to be properly initialized in order for anything to be executed and executed properly. This is where I'm looking for guidance and support from Intel.

I know that you're not suggesting that Intel publishes this documentation with the many complex scenarios and doesn't offer OS developers any help and support when they run into problems? Is Microsoft on their own? Is Apple on their own? Is Linux on their own?

Furthermore, the question about where the BIOS vendors fit into the whole scenario is not being answered. Would I also have to work closely with a BIOS vendor? Or would any motherboard on the market due for development.

Have I made myself clear this time?

Kevin Gregory Smith - Terrytown, La.

[jose-maria-gomez-ver]

You project seems to be very interesting. Do you have any web page or similar to follow up?

[ibtjc4everaol_com]

Quoting josemariagomez

You project seems to be very interesting. Do you have any web page or similar to follow up?

Hi Jose. No, sorry, I don't have any web page for the project. It's just a personal project that I would like to begin work on, but because of obvious industry secrecy and the apparent effort to keep OS Development available only for the elements that matter, I can't move forward with it.


Thanks for the interest.
Kevin Gregory Smith - Terrytown, Louisiana

[TimP]

With about 20 open source operating systems covered on the wikipedia page on the subject, it's difficult to see how the existence of proprietary operating systems could impede your project.