Firstly you will need a paddock for your pony with secure fences, gates and access to running water - preferably a trough with a ballcock that fills itself.
Do a walk around your paddock to ensure all nails/sharp edges are removed from the fences, any rubbish or debris is removed and noxious weeds are removed/sprayed before your pony moves in.

The following items are the basic necessary items for your pony.

Saddle
Bridle
Bit
Stirrups and Stirrup leathers
Saddle Blanket
Girth
Helmet
Jodhpur Boots
Leadrope
Halter
Body Brush
Hoof pick
Mane and Tail Brush
Sponge
Worm Paste
Hay Net
Cover Winter
Cover Summer
Basic Vet Kit - Iodine spray, equifix, black wound powder, vicks, non stick dressing pads, disenfectant wash, cohesive bandages, electrical tape, sizzors, salt, syringe, gamgee, chux clothes.
Sweat Scraper

Other optional items are

Horse Float
Plaiting Bands
Hoof Oil
Grass Rein
jumping/exercise boots
Floating/Travel Boots
Tail Wrap
Sheepskin Numnah
Show Jacket, Shirt, Tie and 'Good' Jodphurs
Show/Travel Rug
Jump Stands and Poles
Mounting Block
Feed Bucket
Hard feed
Lunge Line and Lunge Whip
Crop
Gloves
Comprehensive Vet Kit - Iodine spray, equifix, black wound powder, vicks, non stick dressing pads, disenfectant wash, cohesive bandages, electrical tape, sizzors, salt, syringe, gamgee, chux clothes, saline solution, gloves, thermometer, icepack, poultice, wire cutters, pasternoit or mud fever treatment, coppersulphate, hydrogen piroxide, stockholme tar, gauze cotton wool, glycerine.