Indoor mapping is an essential element of indoor navigation systems. In this paper, we present a graph optimization-based method for indoor map construction, which can be used in buildings without prior knowledge. By using crowdsourcing data on mobile phone sensors, we are capable to derive activity landmarks where people take different activities (turning, taking the elevator, taking the escalator, walking up/down the stairs). Our method uses graph optimization techniques to align crowdsourcing trajectories by their intersecting landmarks. After trajectory alignment, this method applies pose graph optimization method to construct an indoor map. Finally, our method performs a transform from relative coordinates to absolute coordinate with some reference points, and reduce the redundant segments using Dynamic Time Warping (DTW). To evaluate the performance, we implement the proposed method in an office building and a shopping mall. Experiment results show that 80th percentile error of the mapping accuracy is about 1.7-3.5 meters. Moreover, the proposed method can deal with the curved routes in building, and can also decrease the amount of required data.