I find the build mode to be much more intuitive, fun and easy to use. I love being able to move whole rooms! I also really enjoy the way emotions work, becoming focused and inspired, and how they tied that into going to work/school and skill building. Not only does the game run smoother (Sims 3 was such a beast) I think it looks smoother too! I really do't mind the more cartoon-ish look. It works for me.
I don't miss the open world, my little neighborhood gives me enough to explore to keep me happy, and I find the loading time between lots doesn't interrupt my gameplay too much. I was sad when there were no pools, but that's since been remedied! Also, I never really liked the Toddler stage, so I wasn't too sad to see it missing.
I do agree with what you said above though - I constantly find myself hitting 4 thinking it will speed up until the end of the action! I also had to change the camera controls to the Sims 3 way, I couldn't adjust to the new ones.