Debt Free Home Project

Where it started:
Fed up with the insane costs of purchasing a home and disenchanted with the idea of signing over myself over into mortgage debt for the next 30 years, I put the downpayment I had saved for a home into buying a plot of land outright. 0.4 acres in rural NC. Debt free, and house free. But can I change that without entering into building loans and more debt and headaches? Guess we'll see!