India has a glorious history reflected by may heritage sites of architectural and cultural importance. Many of these structures are in partial or fully ruined condition. Digital preservation of a heritage site includes multiple aspects such as in the form of textual description, images, videos, 3D model of a structure etc. We focus on creating a 3D reconstruction framework that enables preserving structures in a dilapidated condition and digitally restores them to their original grandeur. With the application of Augmented Reality, the restored structure can be viewed in situ as a heritage tour. This will be accompanied by an animated playback of a historic event or a animated depiction of a historic artifact.