From personal experience Id say go with software first.
Using software you first first learn what each thing is...
DAW (Digital Audio Workstation) Software
Reason, Pro Tools, Logic, Acid Pro, FL Studio are some examples
The mixing and mastering tools
Reverb, Delay, Compression/Limiting, Expansion, Stereo Expansion/Reduction, EQ'ing, Chorus, Noise Reduction all available as software and hardware, but once you learn how each one works, it doesnt matter if your using either hardware or software, the tools do the same thing, hardware is more hands on, but the principle of each tool remains the same.
You really dont need to worry about hardware until you are proficient in using the tools above in their software versions.
Software will be cheaper, and so would be a better route for a beginner.
And if you find that music production is something you would like to stick with, then you can take your success as an artist and put the money back into some hardware. But that stuff can run into the thousands of dollars.
But then so can software.
The best advice for any beginner of anything, as already stated, "Start Small"
Its best to find out music production isnt for you before you have spent thousands on equipment and not after.
There is some hardware that you will never be able to avoid having to get..
A good pc/mac, Id personally say at least a quad core upto i3,i5,i7.
This is necessary to run all the software so has to be powerful.
Decent Studio Monitors....... These are so damn important.
A good audio interface with a decent output 24bit 192kbps with low latency... This is also very important.
The better the quality of sound that reaches your ears the easier it is to know what needs to be done.
A midi controller is also essential, Id recommend a keyboard at least, and maybe even a MPD controller.
A turntable/cdj, mixer and serato is an optional extra if you intend to add scratches to your tracks.