Evidence for the oldest oceans on Earth is found within minerals, known as zircon, that can be found in Jack Hills, Australia. These zircons have oxygen isotopic compositions suggesting the presence of water on the Earth's surface 4.404 billion years ago, ~150 million years after the Earth formed. These grains do not constrain the volume of water on Earth, so it is hard to speculate on the size of the oceans, but by 3.5 billion years ago abundant water-requiring rocks were widespread.
While the origin of water on Earth is still a "hot topic", we have some ideas of where it might have come from. It is thought that the Earth formed relatively dry and that it lost water during accretion and atmospheric escape. While it has been suggested that comets could have provided the water to form Earth's ocean, isotopic compositions suggest that only 10% of Earth's known water could be cometary. How much water was originally on the Earth is unknown, but today the Earth is around 200 parts per million water. Recent models indicate that less water may have been required during Earth's initial stages of formation, suggesting that planets form with enough water to form an atmosphere, that, upon cooling, collapses into an ocean.
The zircon mineral grains from Jack Hills, Australia, which is the oldest evidence of liquid water on Earth, are also the oldest evidence for continents on Earth. This is not a coincidence - to form siliceous rocks that make up the continents, such as granite, you need water! Subduction of one tectonic plate below another likely played an important role in forming these zircons, but more research is required to confirm this. Subduction produces continental crust (and the zircons) by transferring water into the mantle in the form of hydrous minerals in the down going tectonic plate. When water in these minerals becomes unstable under pressure as the slab descends, water escapes, triggering melting of the rocks above. These melts eventually solidify as the siliceous continental crust.
The rate of crustal production over time is currently debated. The oldest continental rocks are 4.03 billion years old (dated by MIT Professor Samuel Bowring!). The first big continent formed around 2.5 billion years ago, and the average age of continental crust is 2 billion years old. It seems likely that the rate at which crust is produced and destroyed has gradually decreased since the planet formed, preferentially preserving younger rocks, explaining why we only see a small amount of old crustal rocks! More information: