When asking the question "is Yelp reliable?" you're probably asking about one of two things: either the actual information about the businesses listed on Yelp, or the reviews and ratings that people post about businesses on Yelp. In the interest of thoroughness, we'll cover both separately.
Yelp gets most of its information about businesses from third parties, who collect this information from publicly-available sources. Yelp's users can also help with this process, correcting any erroneous information that they see, or adding new information about a business that has flown under the radar until now. Many business owners also list and manage their business's information on Yelp personally.
What does this mean for how reliable the information is? Obviously, information that is supplied by the people who own the businesses on Yelp is going to be accurate. This is, of course, assuming that these people are active on Yelp, or don't change aspects of their business too often.
Beyond that, the contributions of third parties and (more so) of Yelp's users is a bit iffier. The hope is that with more people contributing information, more correct information gets posted, more incorrect information gets noticed and updated or removed, and both of these things happen faster. However, with the number of business listings on Yelp most likely outnumbering the people who provide information services to Yelp on a regular basis, there are bound to be a few incorrect or outdated details somewhere.
The bright side is that the information on Yelp not only is rather specific, but also can be checked against publicly-available sources such as maps and telephone registries. This means that most of Yelp's business information is bound to be fairly accurate, and if it isn't, it's most likely just out-of-date rather than completely false.
Reviews and Ratings
Critiques of businesses that people post on Yelp are most definitely the bigger question mark in terms of Yelp's reliability. The general consensus is that most of the reviews and ratings on Yelp are posted by well-intentioned users who are looking to win social recognition or show off their creative writing skills; to those ends, it's in their interests to be honest. Yelp is also considered to be one of the best websites when it comes to cracking down on businesses that try to cheat the system by writing their own reviews, or paying people to write positive reviews for themselves and/or negative reviews for their rivals.
With that said, there are still rumours and allegations about the number and type of ratings and reviews that show up on the pages of some businesses. There are some small business owners who have accused Yelp's review filter system of causing fewer positive ratings and reviews and/or more negative ratings and reviews to show up on their company's page. They suspect that this is linked to their decision to not purchase an advertising package from Yelp and just continue using Yelp's free features.
However, several legal cases (as well as a few independent studies) have found no evidence linking whether or not a business listed on Yelp buys an advertising package to whether Yelp displays more positive ratings and reviews, or more negative ones, on that business's page. Yelp itself even points out that some of the highest-rated and best-reviewed businesses on Yelp have never purchased advertising packages.
The bottom line, then, is that the majority of ratings and reviews for businesses on Yelp are fairly reliable, being kept in check by both Yelp itself and its users (who can label and comment on reviews as a sort of endorsement or as a way to point out something potentially fishy). If you're ever in doubt, though, the surest way to know is to visit the business yourself and try out its products or services. You might be delightfully surprised, disappointed, or get exactly what you expected. You know your own shopping habits, so you're the best judge as to whether a business is ultimately good or not.