On the fan there are two parts that push in (springy) one at each corner, top left and bottom right. Pressing these in and pulling doesn`t seem to remove the fan. There are also two tiny screws. Not sure how to get this thing off. I do not want to force it.
It depends on the model -- some of them just click/turn on, others are attached with screws to the bottom of the motherboard which would require you to remove your entire motherboard from the computer in order to access it. Keep in mind that the fan/heatsink is a single combo device. Typically you're not just trying to remove the fan, but the entire fan + heatsink combination unit.
There may be a make/model printed on your fan, try googling that to look for the installation instructions.
But like the others say: fans a cheap, CPU's are not. when in doubt, replace the fan. Especially if it's on its way out, you don't want it to go worse and completely grind to a halt. Without a fan, your computers has only minutes to live. (And without a heatsink, we're talking
seconds. so whatever you do, after you remove the heatsink, do NOT turn on your computer. Your CPU will go up in smoke before its past the BIOS screen. (Even though most CPU's have thermal protection these days, without a heatsink the temperature can rise too fast for the protection to even kick in and throttle things back)
Finally: When you replace the fan + heatsink, keep in mind that you'll either need a heatsink that comes with a thermal pad pre-applied, or you'll also need to get some thermal goop like arctic silvergoop like arctic silver
The top of a CPU has microscopic hills and valleys, and for optimal heat transfer it needs the best contact with the heatsink that you can get. The thermal stuff will fill these tiny uneven spots and make sure there are no gaps or air bubbles.
I'd go get a new silent cooling solution, depending on your CPU and case size etc. You can get large heavy heatsinks that require no fan at all.
Note that the fanless systems are not suitable for top-of-the-line CPU's thanks to the extra heat, AND that you need to have sufficient airflow in your case itself to make them work properly.
Rather than going fanless, personally I just looked for a fan that had a low noise level (most fans list their dB, lower is quieter). I also got a fan that worked with my motherboards temperature control feature, so the motherboard could throttle down the speed if the temperatures were low enough. Most of the time mine spins at 300 RPM, and still keeps the temperature below 45C. It's next to inaudiable.