The development of information and communication technologies has produced massive human sensing datasets, such as point-of-interest, mobile phone data, and social media datasets. These datasets provide alternative human perceptions of urban spaces; therefore, they have become effective supplements for remote sensing tasks. This study presents an exploratory framework to examine the scale effect of fusing remote sensing and human sensing. The physical and social semantics are extracted from raw remote sensing images and human sensing data, respectively. A dynamic weighting strategy is developed to explore the fusion of remote sensing and human sensing. Taking urban function inference as an example, the scale effect is evaluated by weighting remote sensing and human sensing. The experiment demonstrates that fusing remote sensing and human sensing enables us to recognize multiple types of urban functions. Meanwhile, the results are significantly affected by the scale.