Exploiting coarse grained parallelism in conceptual data mining: Finding a needle in a haystack as a distributed effort

Mark Blokpoel, Franc Grootjen, Egon van den Broek

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    14 Downloads (Pure)

    Abstract

    A parallel implementation of Ganter’s algorithm to calculate concept lattices for Formal Concept Analysis is presented. A benchmark was executed to experimentally determine the algorithm’s performance, including an AMD Athlon64, Intel dual Xeon, and UltraSPARC T1, with respectively 1, 4, and 24 threads in parallel. Two subsets of Cranfield’s collection were chosen as document set. In addition, the theoretically maximum performance was determined. Due to scheduling problems, the performance of the UltraSPARC was disappointing. Two alternate schedulers are proposed to tackle this problem. It is shown that, given a good scheduler, the algorithm can massively exploit multi-threading architectures and so, substantially reduce the computational burden of Formal Concept Analysis.
    Original languageUndefined
    Title of host publicationProceedings of the 8th Dutch-Belgian Information Retrieval Workshop (DIR2008)
    EditorsE. Hoenkamp, M. De Cock, V. Hoste
    Place of PublicationMaastricht, The Netherlands
    PublisherMaastricht University
    Pages7-13
    Number of pages7
    ISBN (Print)978-90-5681-282-9
    Publication statusPublished - 14 Apr 2008
    Event8th Dutch-Belgian Information Retrieval Workshop, DIR 2008 - Maastricht, Netherlands
    Duration: 14 Apr 200815 Apr 2008
    Conference number: 8

    Publication series

    Name
    PublisherMaastricht University

    Conference

    Conference8th Dutch-Belgian Information Retrieval Workshop, DIR 2008
    Abbreviated titleDIR
    CountryNetherlands
    CityMaastricht
    Period14/04/0815/04/08

    Keywords

    • METIS-252703
    • Formal Concept Analysis
    • IR-79113
    • Parallel
    • Parallelism
    • HMI-IE: Information Engineering
    • Information Retrieval (IR)
    • EWI-20953
    • concept lattices
    • HMI-SLT: Speech and Language Technology
    • Cranfield collection

    Cite this

    Blokpoel, M., Grootjen, F., & van den Broek, E. (2008). Exploiting coarse grained parallelism in conceptual data mining: Finding a needle in a haystack as a distributed effort. In E. Hoenkamp, M. De Cock, & V. Hoste (Eds.), Proceedings of the 8th Dutch-Belgian Information Retrieval Workshop (DIR2008) (pp. 7-13). Maastricht, The Netherlands: Maastricht University.