I spent more than 21 (at the time of write this post) working on I.T. and I have the opportunity of being involved as early adopter and researcher of cutting edge technology that later becomes a standard and still can remember things like:
Can sounds crazy now, but remember than things like Cloud providers, Docker, SDN, SDWAN.... just didn't exist or being research projects.
Time goes fast and in technology flies and I was so luckily to work with those disruptive technologies from their born to what are now in many different roles and organizations.
One of the most valuable thing on my career it's the big number of IT people I had opportunity to engage with. My favourite conversation usually start when a "non believer" approach you saying:
Well, the reality is that something must being doing right the main CSPs (AWS, GCP, OCI, Azure) when Cloud market is growing year by year. But, what they offer to catch the attention of organizations?, why is so difficult to build something similar ?.
Lets start from the beginning
The business case:
Each cent you plan to spend in IT must have a business case, no matter if is your homelab (where you plan develop new skills than will let you become a better paid professional), an startup (straggling with the need of raise funds (each cent counts), a big company (running a private or hybrid Cloud) or a Cloud Service provider.
Your business case will define the objectives and IT will translate it to pieces of technology like services, features, etc to bring value to the business.
Build a clear business case with clear objectives is the foundational stone, before start installing the last Linux distro to run Open Stack, Cloud Stack. VMware Cloud... or whatever you plan to use. Maybe you don't need as many (and as complex) services like one of the big four. Think about what you really need and try to keep it simple. Simplicity is the way to other goals like reliability, security, etc.
The Services:
The number or services can be almost endless, but keeping in mind simplicity you can start thinking on provide at least :
You need compute to run your vm instances. In the beginning of cloud days most of the providers start using Xen, later other software become more mature and used and currently KVM is the leader Hypervisor.
You need Storage to make your data persistent and ideally two types of storage one block based and other object based.
You need network to make your services available
You need IAM to manage users and roles
and finally you need monitoring capabilities. Of course there are other services than can be considered foundational (depend of your use case), but this is a minimum set of services.
The software
That's another question that people usually raise because they're curious about what software use AWS, GCP, OCI or Azure to run a cutting edge Cloud.... and there is no simple answer for that question :-), but I can tell you that most of the services can be implemented using software available for everyone. For example, the Hypervisor, VMware, Xen, KVM.. are available for everyone and most of them are Open Source (except VMware).
Probably the main question is, do you want to go with a proprietary solution (because the support, software limitations, etc.)? or try to do it yourself using a Open Source project.
In the proprietary side you can find between others:
if you prefer be free as a bird them:
probably there are others, but those cover almost 100% of deployments.
Network or how to connect to your cloud
Your new cloud services must me reachable by some way and that is the network. Typically in a CSP services will be accessible through public internet (if you publish them), via IPSec VPN or by dedicated connection.
Use the public internet or a VPN is not rocket science if we talk about dedicated connection you will need to find a provider that connect your remote locations to your cloud datacenters, depend of that location you will have more or less options and prizes. If we pretend also to has dedicated connectivity with one or more of the big 4 providers, then you will need in addition a dedicated link to one of the PoC of those providers.
Big CSPs not always run their own facilities (remember simplicity principle), in many occasions they're collocated in specialized providers liker Equinix, Digital Realty, etc.. If you collocate your cloud in one of those providers, then can user the provider backbone to reach other regions or other CSPs collocated in the same region.
Muolticloud is the future of cloud!!.
Deal with complexity
Don't underestimate how complex can be run something like a CSP and most important don't pretend to be AWS, GCP , OCI or Azure. They are big corporations with huge budgets and an army of engineers, SREs, developers, support teams, field engineers, a footprint of datacenters globally distributed... What I mean is, only because the big four has amazing services like serverless, Database PaaS, BigData, Managed Kubernetes, etc. you don't need to build it id there is no reason to do that (remember the business case).
If you need a service you can't provide, maybe the cost of consume it from a external provider will be smaller than the cost and effort needed to run it yourself.
Multicloud ready
In the beginning there was only AWS and all other vendors deal with this situation, fighting to get part of the AWS (current or potential) market share. Choose your cloud provider was like choose your football team and vendors invest a lot of in build communities of fan boys :-).
Today situation is becoming different and is no unusual find organizations running a few services on provider A and other in provider B, with both clouds connected. Find what you do great and integrate with others that can bring you the value you business need and you're not ready to deliver yourself.
A perfect example of that approach can be found in the multi cloud strategy from Oracle OCI. They provide some of their key services to other providers like AWS, Azure or GCP so users of those providers can consume OCI services like any other native service from their provider or through high performance and resilience connectivity.