The most important responsibility for every recommendation system is how to make the appropriate personalized recommendation for different customers rapidly and effectively. Collaborative filtering recommendation is one of the most popular methods among the E-commerce system, but it still remains some problems, such as "cold start" and "data sparse". At the same time, more and more users registering on make the real-time and expansibility of a system hard to be kept. In the paper, according to a series of problems, such as "data sparse" and bad "real-time" caused by a large number of registering users, we propose a new approach combining the clustering algorithm with SVD algorithm which is widely used in the domain of image-processing into collaborative filtering algorithm. Firstly, we classify the users by using the attributes of them. Then we decompose the rating matrix with the SVD algorithm and reunion them into new rating matrix to calculate the similarity between each pair of users. At last, we take advantage of the similarity to find the nearest neighbors in the collaborative filtering recommendation and predict the ratings of the items to make the recommendation. Our experiments show that the approach cannot only improve the "cold start" and "data sparse" problems but also increase the efficiency and scalability of a system.