There several ways to do so. I recommend using JQuery:
Let's suppose you have the html code of your side bar in sidebar.html file.
Then, in any of the html files you want to include, you should create a div for containing it and load the sidbar.html content in it:
<html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script> <!-- ...or whereever you have jquery stored -->
<script>
$(function(){
$("#sidebar").load("sidebar.html");
});
</script>
</head>
<body>
<!-- other content here -->
<div id="sidebar"></div>
<!-- other content here -->
</body>
</html>