Of Dairy and Darkness is a little, uh, weird.
The Witcher 3: Of Dairy and Darkness
You can stumble across the main location of this quest while exploring the wilds around Novigrad, but even if you have done so you'll need to make a visit to the merchant across the street from the Gildorf Square Noticeboard (near the St Gregory's Bridge signpost) to kick things off properly.
This enchanting fellow, who has a Gwent deck full of spies, is in possession of a number of items once owned by the executed mage Aeramas. Purchase the Lizard Figurine to proceed (you can also grab the Jade Figurine to start another quest).
With the lizard in hand proceed to Aeramas's estate, which should be marked by a waypoint if you're tracking this quest. It's kind of in the middle of nowhere; a long way southeast of the Temerian Hideout, quite a distance northeast of the Crossroads signpost.
Climb into the ruins of the manor and use Witcher Senses to locate a magical circle on the floor. Activate it (this is why you needed the lizard) to open a portal to the mage's lair.
Inside, you need to pass through a field of cheese. Yes, I know. Your goal is a ruined staircase on the far side which you can climb onto - but to get there you need to pass through the deadly minefield of cheese. Step into the cheese field and look around at each junction for a path not marred by toxic cheese fumes. If you step into the fumes, step back quickly - the Killer Whale potion can help you survive a little longer here, but not much. By picking your way through the safe spots you should eventually reach the stairs. Climb up and follow the path to the next room.
In the second area you'll have to do some swimming. The east path is a dead end, so go north where you can climb out onto dry ground. You need to climb and leap yourway around the higher reaches of this chamber until you reach Aeramas's workshop across a broken bridge. Search around, pausing to find and read the mage's notes if you like, until you find another figurine. use this to activate a portal in the corner, and pass through.
In the third and final room you'll see a magical barrier. Use your Eye of Nehaleni, obtained during Wandering the Dark in Act One: Velen (No Man's Land), to dispel the illusion masking an object on the left side of the room. Cast Igni on the crystal you reveal, and cast it on the other crystal on the opposite side of the room. The barrier retracts.
Loot the treasure chest and the statue, obtaining a final figurine as well as a nice sword, then activate the last portal and appear back in the ruins.