"Solar" refers to the Sun, while "Sidereal" refers to the stars.
The Earth is doing two things at once; rotating on its axis, and traveling along the path of its orbit around the sun. It takes the Earth 23 hours 56 minutes (and a few seconds) to make one 360 degree turn on its axis; at night when looking at the stars, you would see the stars appear to travel across the sky, set in the west, rise in the east, and come back to their same positions in the sky. That's one sidereal day, referenced to the stars.
But during that time, the earth has continued in its orbit around the Sun, and it will take another four minutes for the Earth to rotate enough so that the Sun would come back to the same position in the sky. That's a solar day.