Question regarding a domain name change

I've had my current domain name for i'd say about 5 months.

I'd like to make it a little more niche specific, atm it is not.

What are the drawbacks to this?

Will there be an immediate drop in serps?

I'm assuming all backlinks will be lost too?

Anytime you move something, expect a drop in serps and re-index period.

The hit NOW is likely better to take than the hit LATER, but it still has to happen.

Depending on where you go, you can likely set a global redirect to keep old links in tact and pass 301's to the search engines to improve re-index response time.