This was a deliberate decision, because we wanted to minimise overlapping behaviour of both media managers. While some of the old behaviour still needs to be maintained in both (like upload) because it's a frequent task, we felt that deleting something is not as frequent and can go solely into the new media manager (page, not toolbar). It makes sense to put all "managing functionality" only into the fullscreen media manager and as few as possible into the popup media manager.
I already missed the delete link myself as well. So maybe we should readd it. The popup is also missing a link to open an image in the fullscreen manager.