EX FRIENDS W/BENEFITS ADVISE NEEDED!?

Need help from guys 25 or older or women who've had similar experiences. I had a fwb situation w/a guy for a while. He vanished, but then started talking to me again several months later. We sort of hooked up again a few time, but again the fwb aspect stopped. We have remained friends though this time, we text and talk, and he's even hung out with me. But here the problem. When we were fwbs, he had problems staying hard for . I took it personally, that he just wasn't into having w/me and I even said so to him. I still bring it up now and then, but he never answers me. The thing is, he's tried to get me to perform oral on him now, but I won't. I believe I deserve to get what I want too, so why should I just give him what he wants. I was never critical to him about his problem, I did what I could to help the situation. But I don't know if it is really me or not. I'm not the most attractive person and I'm a plus size gal, so I don't know I'm just not attractive enough for him? (note: I've had plenty of guys who seemed to be into me and enjoy being with me without any problems). But then why would he have still come over as much as he did. He did put alot of effort into giving me what I wanted (not just oral but actual ) at times. So how do I know now if the problem is with him or with me? When we hung out, he was suddenly into some aspects of me. I know he's not looking for committment, but either am I. We actually are on the same page as to what we would want in some aspects. Nothing serious, just fun and frienship. He says all guys only want oral and won't stick around for long. But yet he tells me about wanting with other women. I'm very confused and don't know what to think! Any ideas, opinions, experiences, or suggestions?