I could not sleep last night and was thinking about this, you might be in luck. Their nests are not like most wasps nest. They have no workers and queen and such, if I remember right. Those are just brooding nest, the wasp, as I said before, build little mud tubes and lay a single egg in the bottom. Then they pack it with spiders to feed the grub when it hatches. Then the adult leaves and leaves the nest to itself. It gets hard as it drys and just sets there as the egg hatches and matures into a single wasp.
I am not sure but I have broken them open, none as big as that one, and they are just grubs. It seems that there would be little danger to you trying to remove it except for a few that might be about ready to hatch out. I think I would spray the sucker down good with a can of wasp spray. They will shoot 20 ft, just soak it down and then let it set for a while. Then I think you could probably just go back with a putty knife and pop the whole thing off in big chunks. An adult wasp would have been killed by the spray and the larva are little danger to you.
I think that is what I would try. You are lucky they are not paper wasp.
I would also find the opening they use to get back in there and seal it up.
Good luck