Even finally acting on her feelings for Dyson was because there was no Bo to prevent it.
You may be right. I've got used to Dyson being totally devoted to and in love with Bo and I don't remember Kenzi ever implying to have romantic feelings for him either. If I had been able to look at their make out session without their previously established emotional states in my mind, then yes, perhaps it wouldn't have felt so unpleasant. Thanks to Fae gods, everything went back to normal in the end, so I guess I don't have to worry about unexpected kisses between them anymore.
It always seemed to me that the main reason that Kenzi wanted Bo and Dyson together so badly (aside from her issues with Lauren) was so she could sort of have a romantic relationship with him vicariously through Bo. Her obsession with his 'private parts' was I suppose the biggest clue I had about that, but the whole thing always felt that way to me, ever since season one. To me the only surprising part of seeing them together was the extent of Dyson's feelings for Kenzi, he had sort of vaguely hinted from time to time in the past, that maybe he had some feelings for her, but that was about it. He's been so obsessed with Bo he's never really let on about his feelings for Kenzi, so it was surprising to see just how deeply he felt about her, and it does suggest the possibility that perhaps Dyson will at some point 'finally' get over Bo and move on to Kenzi (something that seeing Bo with Tamsin would no doubt help him do ) . It's something I'd like to see happen anyway .