To start, check the small wire on your starter solenoid when the starter stays engaged. If you have power to this wire with the key off, then the problem lies farther up stream like the starter key switch. As OJL mentioned, you can use a starter push button switch and connect to the battery side of the solenoid and the small terminal on the solenoid. If the starter engages and disengages when you push the button, the problem also lies upstream of the solenoid. You can also cross these 2 terminals with a screwdriver if you don't have a switched jumper with a button. I have had a couple of times where I replaced the solenoid with a new one only to have the same problem. I eliminated all other possibilities, and found out that the new solenoid was defective. You have to remember that the only way for power to get to the starter is through the solenoid, so if the starter is staying engaged, it is getting power through the solenoid, not the starter itself. My feeling is that you just got a bad solenoid.