My best guess is the Tales to Tremble By scary story collections:

Each story had some creepy art by Gordon Laite:

A lot of this fits with your descriptions from the best of my recollection.

Edit: Removed paragraph about potential third scary story series containing spider egg story, just realized that one was from Scary Stories to Tell in the Dark 3, so Tales to Tremble By and its sequel is my only guess here.

Was there a story about a girl and a ribbon in the book you read, and when the ribbon was removed her head fell off?

If so it might have been “In a Dark, Dark Room” but that was published in the 80s

edit: I’d also look at Ed Gorey as well, his art style is in that same vein. Also i can’t read

After some unproductive googling, here's a random guess. The cover art style doesn't really match (not sure if the inside is different), but it has a deep sea diver and a lady underwater, who I can only assume is the "Ghost Queen"

I feel like I've seen those books way back in the day but I couldn't tell you the name of them to save my life.

Can't help with an answer, but you could also try posting this over on !Tipofmytongue@lemmy.world, might get a bite.

You didn't mention Scotland, but on the off chance, could it have been Ghosts Go Haunting?

I haven't even read your post, just copy pasted it into chatgpt o1 preview model and it said this:

Based on your detailed recollections, I believe the book you're searching for may be one of the anthologies by Ruth Manning-Sanders, particularly those illustrated by Robin Jacques. Here’s why:

1. Ruth Manning-Sanders' Anthologies:

"A Book of Ghosts and Goblins" (1968): This collection features a variety of eerie tales from around the world, all retold by Manning-Sanders. The stories often involve witches, ghosts, and other supernatural beings.

"A Book of Witches" (1965): This anthology includes tales specifically about witches, which might correlate with your memory of a sea-witch draped in seaweed.

"A Book of Mermaids" (1967): Given the sea theme and possible illustrations of underwater scenes, this book could also be a contender.

2. Illustrations by Robin Jacques:

Art Style: Jacques is renowned for his intricate pen-and-ink drawings with heavy linework and detailed crosshatching, which matches your description.

Depictions: His illustrations often have a gothic and eerie feel, featuring classic elements like willow trees, cliffside scenes, and supernatural creatures.

Sample Illustrations:

Diver and Sea-Witch: While I can't access specific images, Jacques' work frequently includes mythical and maritime elements that could involve divers and sea-witches.

Creepy Willow Trees and Gothic Cliffs: His landscapes are detailed and atmospheric, possibly aligning with the images you remember.

3. Physical Attributes:

Binding: Many editions of Manning-Sanders' books from the 60s and 70s came with green or teal bindings, especially after losing their dust jackets.

Publication Era: The books were published in the mid to late 60s, fitting your timeframe of "old but not ancient."

Age Appropriateness: These books were marketed as children's literature but often contained darker themes and illustrations that could slip past less attentive parents.

4. Why This Might Be the Book:

Combination of Elements: The combination of maritime tales, witches, and eerie illustrations matches the key elements you recall.

Illustrator's Style: The pen-and-ink crosshatching and gothic atmosphere in Jacques' illustrations seem to align closely with your memories.