We performed whole exome or genome sequencing in eight multiply affected families with ostensibly isolated congenital anosmia. Hypothesis-free analyses based on the assumption of fully penetrant recessive/dominant/X-linked models obtained no strong single candidate variant in any of these families. In total, these eight families showed 548 rare segregating variants that were predicted to be damaging, in 510 genes. Three Kallmann syndrome genes (FGFR1, SEMA3A, and CHD7) were identified. We performed permutation-based analysis to test for overall enrichment of these 510 genes carrying these 548 variants with genes mutated in Kallmann syndrome and with a control set of genes mutated in hypogonadotrophic hypogonadism without anosmia. The variants were found to be enriched for Kallmann syndrome genes (3 observed vs. 0.398 expected, p = 0.007), but not for the second set of genes. Among these three variants, two have been already reported in genes related to syndromic anosmia (FGFR1 (p.(R250W)), CHD7 (p.(L2806V))) and one was novel (SEMA3A (p.(T717I))). To replicate these findings, we performed targeted sequencing of 16 genes involved in Kallmann syndrome and hypogonadotrophic hypogonadism in 29 additional families, mostly singletons. This yielded an additional 6 variants in 5 Kallmann syndrome genes (PROKR2, SEMA3A, CHD7, PROK2, ANOS1), two of them already reported to cause Kallmann syndrome. In all, our study suggests involvement of 6 syndromic Kallmann genes in isolated anosmia. Further, we report a yet unreported appearance of di-genic inheritance in a family with congenital isolated anosmia. These results are consistent with a complex molecular basis of congenital anosmia.