Four days is the sweet spot for Amelia Island, because it is the first length at which you can do the island properly and take the ferry to Cumberland Island.
The organising rule
Group by geography, not by interest. The island is thirteen miles long, and planning by category — "history day", "beach day" — means driving its length repeatedly. Planning by area does not.
The four blocks: north end, downtown, oceanside, south end. Roughly one a day, with the ferry taking one of them out.
Day 1: the north end
Sunrise first, at any Fletcher Avenue beach access. The island faces east; at 6.30 in summer you will have the beach to yourself.
Fort Clinch State Park at opening. The 1847 masonry fort — ramparts, galleries, kitchens — at the end of a mile of live-oak canopy. Two beaches and a fishing pier in the same park, plus a six-mile off-road bike trail. Half a day easily.
Old Town Fernandina on the way back: the 1811 Spanish plat, the Plaza San Carlos, and Bosque Bello Cemetery under live oaks. Forty minutes, almost unvisited.
Dinner downtown, and sunset on the marina — downtown faces west across the Amelia River, so this is where the day ends.
Day 2: Cumberland Island
Book this before you travel. The ferry leaves from St Marys, Georgia, about 35 minutes north via A1A and I-95. Reservations are required and it books out, especially in spring.
Eighteen miles of undeveloped beach, maritime forest under Spanish moss, the burnt Dungeness ruins, and wild horses — feral and unmanaged, so keep well back.
There are no shops. You carry your food and water in and your rubbish out. It is a full day: ferry over, several hours walking, ferry back, and know your return time before you start walking north.
Best day trip in northeast Florida or southeast Georgia, and worth the logistics.
Day 3: the marsh and downtown
Morning — a guided kayak trip into the salt marsh behind the island. The island's best experience and the one most visitors miss: tidal creeks, dolphins, wading birds, and a landscape invisible from any road. The operator plans around the tide, which is what ruins independent paddling here.
If paddling is not for you, the Egans Creek Greenway does the same landscape on foot, free, with alligators visible from the trail. First light.
Afternoon — downtown properly. Eight blocks of Centre Street and the cross streets, the independent shops, the Amelia Island Museum of History for the Eight Flags story, and the working shrimp docks.
Dinner at one of downtown's upper-end rooms. Book it.
Day 4: the south end
American Beach and Burney Park. The community founded in 1935 by the Afro-American Life Insurance Company so that Black Floridians could reach an ocean beach under segregation, with NaNa, Florida's tallest dune, protected behind it, and the A.L. Lewis Museum nearby. Twenty minutes that change how the rest of the island reads.
Amelia Island State Park at the southern tip: undeveloped sand, shelling on the sound flats, the George Crady Bridge fishing pier, and — if you booked it — beach horseback riding, the only Florida state park that permits it.
Or a proper beach afternoon at Peters Point, which has the room and the facilities.
Last dinner downtown or, for a different register, a resort room on the south end.
What to book before you travel
In this order: the Cumberland ferry (it constrains everything else), beach horseback riding, the kayak trip, and dinner for your Friday or Saturday.
Where to stay
Oceanside Fletcher Avenue is the best base for four days: you walk to the sand and downtown is five to eight minutes away, so no day involves a long drive. Downtown works if walking to dinner matters more than walking to the beach.
A south-end resort costs you time on this itinerary — you would drive north on three of the four days.
If you have five days
Add Big and Little Talbot Islands and Kingsley Plantation to the south, which make a good half-day with a Jacksonville stop. Or St Augustine, about ninety minutes south and a completely different era of Florida history.
The weather caveat
Hold a wet-day plan. The island has almost nothing indoors — the museum, the shops, a long lunch — and a genuinely wet day is best spent driving forty minutes to Jacksonville. On a four-day trip in winter or spring, you may well need it once.