Furniture style compatibility recommendation with cross-class triplet loss

Tse Yu Pan, Yi Zhu Dai, Min-Chun Hu, Wen Huang Cheng

Research output: Contribution to journalArticle

Abstract

Harmonizing the style of all the furniture placed within a constrained space/scene is an important principle for interior design. In this paper, we propose a furniture style compatibility recommendation approach for users to create a harmonic 3D virtual scene based on 2D furniture photos. Most previous works of 3D model style analysis measure the style similarity or compatibility based on predefined geometric features extracted from 3D models. However, “style” is a high-level semantic concept, which is difficult to be described explicitly by hand-crafted geometric features. Moreover, analyzing the style compatibility between two or more furniture belonging to different classes (e.g., table and lamp) is much more challenging since the given furniture may have very distinctive structures or geometric elements. Recently, deep neural network has been claimed to have more powerful ability to mimic the perception of human visual cortex, and therefore we propose to analyze style compatibility between 3D furniture models of different classes based on a Cross-Class Triplet Convolutional Neural Network (CNN). We conducted experiments based on a collected dataset containing 420 textured 3D furniture models. A group of raters were recruited from Amazon Mechanical Turk (AMT) to evaluate the comparative suitability of paired models within the dataset. The experimental results reveal that the proposed furniture style compatibility method based on deep learning performs better than the state-of-the-art method and can be used to efficiently generate harmonic virtual scenes.

Original languageEnglish
Pages (from-to)2645-2665
Number of pages21
JournalMultimedia Tools and Applications
Volume78
Issue number3
DOIs
Publication statusPublished - 2019 Feb 1

Fingerprint

Electric lamps
Semantics
Neural networks
Experiments
Deep neural networks
Deep learning

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Cite this

Pan, Tse Yu ; Dai, Yi Zhu ; Hu, Min-Chun ; Cheng, Wen Huang. / Furniture style compatibility recommendation with cross-class triplet loss. In: Multimedia Tools and Applications. 2019 ; Vol. 78, No. 3. pp. 2645-2665.
@article{8c0dd34918d540efa1a6d13a76815483,
title = "Furniture style compatibility recommendation with cross-class triplet loss",
abstract = "Harmonizing the style of all the furniture placed within a constrained space/scene is an important principle for interior design. In this paper, we propose a furniture style compatibility recommendation approach for users to create a harmonic 3D virtual scene based on 2D furniture photos. Most previous works of 3D model style analysis measure the style similarity or compatibility based on predefined geometric features extracted from 3D models. However, “style” is a high-level semantic concept, which is difficult to be described explicitly by hand-crafted geometric features. Moreover, analyzing the style compatibility between two or more furniture belonging to different classes (e.g., table and lamp) is much more challenging since the given furniture may have very distinctive structures or geometric elements. Recently, deep neural network has been claimed to have more powerful ability to mimic the perception of human visual cortex, and therefore we propose to analyze style compatibility between 3D furniture models of different classes based on a Cross-Class Triplet Convolutional Neural Network (CNN). We conducted experiments based on a collected dataset containing 420 textured 3D furniture models. A group of raters were recruited from Amazon Mechanical Turk (AMT) to evaluate the comparative suitability of paired models within the dataset. The experimental results reveal that the proposed furniture style compatibility method based on deep learning performs better than the state-of-the-art method and can be used to efficiently generate harmonic virtual scenes.",
author = "Pan, {Tse Yu} and Dai, {Yi Zhu} and Min-Chun Hu and Cheng, {Wen Huang}",
year = "2019",
month = "2",
day = "1",
doi = "10.1007/s11042-018-5747-5",
language = "English",
volume = "78",
pages = "2645--2665",
journal = "Multimedia Tools and Applications",
issn = "1380-7501",
publisher = "Springer Netherlands",
number = "3",

}

Furniture style compatibility recommendation with cross-class triplet loss. / Pan, Tse Yu; Dai, Yi Zhu; Hu, Min-Chun; Cheng, Wen Huang.

In: Multimedia Tools and Applications, Vol. 78, No. 3, 01.02.2019, p. 2645-2665.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Furniture style compatibility recommendation with cross-class triplet loss

AU - Pan, Tse Yu

AU - Dai, Yi Zhu

AU - Hu, Min-Chun

AU - Cheng, Wen Huang

PY - 2019/2/1

Y1 - 2019/2/1

N2 - Harmonizing the style of all the furniture placed within a constrained space/scene is an important principle for interior design. In this paper, we propose a furniture style compatibility recommendation approach for users to create a harmonic 3D virtual scene based on 2D furniture photos. Most previous works of 3D model style analysis measure the style similarity or compatibility based on predefined geometric features extracted from 3D models. However, “style” is a high-level semantic concept, which is difficult to be described explicitly by hand-crafted geometric features. Moreover, analyzing the style compatibility between two or more furniture belonging to different classes (e.g., table and lamp) is much more challenging since the given furniture may have very distinctive structures or geometric elements. Recently, deep neural network has been claimed to have more powerful ability to mimic the perception of human visual cortex, and therefore we propose to analyze style compatibility between 3D furniture models of different classes based on a Cross-Class Triplet Convolutional Neural Network (CNN). We conducted experiments based on a collected dataset containing 420 textured 3D furniture models. A group of raters were recruited from Amazon Mechanical Turk (AMT) to evaluate the comparative suitability of paired models within the dataset. The experimental results reveal that the proposed furniture style compatibility method based on deep learning performs better than the state-of-the-art method and can be used to efficiently generate harmonic virtual scenes.

AB - Harmonizing the style of all the furniture placed within a constrained space/scene is an important principle for interior design. In this paper, we propose a furniture style compatibility recommendation approach for users to create a harmonic 3D virtual scene based on 2D furniture photos. Most previous works of 3D model style analysis measure the style similarity or compatibility based on predefined geometric features extracted from 3D models. However, “style” is a high-level semantic concept, which is difficult to be described explicitly by hand-crafted geometric features. Moreover, analyzing the style compatibility between two or more furniture belonging to different classes (e.g., table and lamp) is much more challenging since the given furniture may have very distinctive structures or geometric elements. Recently, deep neural network has been claimed to have more powerful ability to mimic the perception of human visual cortex, and therefore we propose to analyze style compatibility between 3D furniture models of different classes based on a Cross-Class Triplet Convolutional Neural Network (CNN). We conducted experiments based on a collected dataset containing 420 textured 3D furniture models. A group of raters were recruited from Amazon Mechanical Turk (AMT) to evaluate the comparative suitability of paired models within the dataset. The experimental results reveal that the proposed furniture style compatibility method based on deep learning performs better than the state-of-the-art method and can be used to efficiently generate harmonic virtual scenes.

UR - http://www.scopus.com/inward/record.url?scp=85041915556&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041915556&partnerID=8YFLogxK

U2 - 10.1007/s11042-018-5747-5

DO - 10.1007/s11042-018-5747-5

M3 - Article

VL - 78

SP - 2645

EP - 2665

JO - Multimedia Tools and Applications

JF - Multimedia Tools and Applications

SN - 1380-7501

IS - 3

ER -