Fertility is Declining, Should we be Worried?

With fertility on the decline, this should be a concern to all of society. First, why is it declining is the first question that comes to my mind. Then, is there a trend of certain people where fertility is declining? For instance, could it be a lifestyle issue or a cultural issue or is it across the board where fertility is on the decline? If it is across the board, then what has changed? Is it technology related, genetics, food sources, or what could it be? This question is very worrisome and I question why more people are not concerned or voicing their opinions.