In recent years, the huge amount of information and users of the internet service, it is hard to know quickly and accurately what the user wants. This phenomenon leads to an extremely low utilization of information, also known as the information overload problem. Traditionally, keywords are used to retrieve images, but such methods require a lot of annotations on the image data, which will lead to serious problems such as inconsistent, inaccurate, and incomplete descriptions, and a huge amount of work. To solve this problem, Content Based Information Retrieval (CBIR) has gradually become a research hotspot. CBIR retrieves picture objects based entirely on the content. The content of an image needs to be represented by features that represent its uniqueness. Basically, any picture object can be represented by its specific shapes, colors, and textures. These visual characteristics of the image are used as input conditions for the query system, and a result the system will recommended nearest images and data set. This research designs and implements two-stage deep learning-based model that recommends a clothing fashion style. This model can use deep learning approach to extract various attributes from images with clothes to learn the user’s clothing style and preferences. These attributes are provided to the correspondence model to retrieve the contiguous related images for recommendation. Based on data-driven, this thesis uses convolutional neural network as a visual extractor of image objects. This experimental model shows and achieves better results than the ones of the previous schemes.