There is likely nothing wrong with that SSD, nor is there any problem with the way Windows has mounted it.
Windows OS normally mounts small USB drives, e.g., flash "thumb" drive, without write caching to allow for quick ejection.
Large HDD and SSD drives, whether internal or external, are normally mounted with write caching, which speeds up writing to disk by buffering writes in RAM until they are completed. The disadvantage is that one must wait for all writing to complete before removing the drive, and for that reason, an eject button may not be displayed.
If you want to dismount the drive frequently, and you don't mind slowing write access, you can change the drive mount to remove caching.
In Explorer, right-click on the drive and select Properties.
On the Hardware tab, select Properties (again).
Click the Change settings button.
On the Policies tab, remove the check from *Enable write caching on the device". The SSD should now be able to be ejected as would a flash drive.
On the other hand, one can use a third-party tool to safely eject even a drive mounted with write caching. Free HotSwap! and many alternatives can be used, and I find them more reliable.
As for the $RECYCLE.BIN file, yes, this is normal, since it allows for temporary "deletion" of files by moving them to the Recycle Bin. That is usually an advantage of having a large external HDD, but if you don't want to use it, you can set the policy for that drive to "Do not move deleted files to the Recycle Bin".
Right-click the Recycle Bin.
Select the drive letter for the external SSD.
Select "Don't move files to the Recycle Bin..."