The question you are asking has two good answers.
The first simple answer for your level of HTML knowledge would be to use a text-editors Find & Replace functionality to simply change identical pieces of code so all pages match. This would be the simplest answer for your question.
The larger answer would be to introduce you to the idea of a layout file. With this layout file(s) you could define your entire page layout, links, headers, etc. Then with each page you load you simply call the content you want to show inside the layout file. This is a high level concept and should ultimately be your goal to reduce issues like those which you stated in your question.
You can accomplish this with a variety of technologies such as PHP, ASP.Net, or Java EE development which are all very advanced stacks. The best starting point I would suggest is Server Side Includes. This will allow you to simply call the required layout text. While this solution is rather legacy; it might help you accomplish your task with grater efficiency.