Certain network applications on your LAN can create poor VoIP call quality by saturating your local networks router/firewall device. Bandwidth saturation rarely is the cause of this. High local packet per second load, or PPS load, will often push conventional consumer router/firewalls to their limit and not process packets quickly enough or just drop them.
VoIP in itself inherently creates a higher PPS load then regular web/data traffic. A single VoIP phone call is typically 30-50 PPS depending on the codec in use. Other applications that generate a high PPS load are Bit-torrents, Peer to Peer file-sharing & network games such as Battlefield. With all of these applications including your active VoIP calls requesting routing, your router often will start to lag in routing/processing VoIP packets or even drop them, thus resulting in choppy/poor call quality.
QoS can somewhat improve quality, but it is often dependent on how it is implemented on your router/firewall and if it has the processing power to do QoS 'quick' enough. Often, conventional consumer routers such as Linksys/D-Link/Netgear, have marginal QoS ability and still may not improve things much if you require high PPS applications to be running constantly.
VoIP calls are real time IP applications and need to be handled as such at the network level. Factors such as packet loss are very noticeable in real time applications, unlike data downloads which request lost packets to be sent again. Packet loss as little as 1% will be noticeable on a VoIP phone call, where that is often acceptable for data oriented Internet use.