Extraction of knowledge points and relations from learning materials

Field

The embodiments discussed herein are related to extraction of knowledge points and relations from online learning materials.

Background

Open education resources generally refer to online learning programs or courses that are made publicly available on the Internet or other public access networks. Examples of open education resources may include e-learning programs, Open Courseware (OCW), Massive Open Online Courses (MOOC), and the like. Participation in an open education program typically allows a learner to access learning materials relating to a variety of topics. The learning materials may include lecture notes, course syllabus, example problems, lecture video recordings, and the like.

Various open education resources are currently offered by a number of educational institutions. The number of educational institutions offering open education resources has increased substantially since the inception of open education a little over a decade ago. With the proliferation of open education resources, there has been a concomitant increase in the number of available learning materials available online.

The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one example technology area where some embodiments described herein may be practiced.

Summary

According to an aspect of an embodiment, a method of automated domain knowledge structure generation includes crawling, by one or more processors, electronic learning materials stored at least temporarily in one or more non-transitory storage media. The method may include extracting, by the one or more processors, structural information from the electronic learning materials. The method may include extracting, by the one or more processors, knowledge points from the electronic learning materials. The method may include inferring, by the one or more processors, hierarchy and dependency relationships between the knowledge points. The method may include aligning, by the one or more processors, one or more of the knowledge points with one or more of the learning materials. The method may also include generating, by the one or more processors, a domain knowledge structure. The domain knowledge structure may include the extracted knowledge points organized at least partially according to the inferred hierarchy and dependency relationships. The extracted knowledge points may include the aligned learning materials.

The object and advantages of the embodiments will be realized and achieved at least by the elements, features, and combinations particularly pointed out in the claims.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.

Brief description of the drawings

Example embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

FIG. 1 illustrates a block diagram of an example personalized learning environment in which some embodiments described herein may be implemented;

FIG. 2 is a block diagram of an example source analysis server that may be implemented in the personalized learning environment of FIG. 1 ;

FIG. 3A illustrates an example online learning material that may be analyzed in the personalized learning environment of FIG. 1 ;

FIG. 3B illustrates another example online learning material that may be analyzed in the personalized learning environment of FIG. 1 ;

FIG. 3C illustrates another example online learning material that may be analyzed in the personalized learning environment of FIG. 1 ;

FIG. 4 is a flow diagram of an example method of domain knowledge structure generation;

FIG. 5 is a flow diagram of an example method of structural information extraction;

FIG. 6 is a flow diagram of an example method of knowledge point extraction; and

FIG. 7 is a flow diagram of an example method of dependency inference.

Description of embodiments

Learning materials available online have increased of due to the proliferation of open education resources. For example, each course included in an open education resource may include learning materials such as videos, lecture notes, transcripts, test questions, a syllabus, etc. Thus, manual organization (e.g., individuals analyzing and structuring various lecture slides, videos, etc.) of the learning materials has become increasingly difficult.

In addition, the learners who use open education resources may have difficulties finding information related to specific concepts and ascertaining relationships between a specific concept and related concepts. Some educational programs provide learners with manually created organizational structures. These structures may include broad concepts with embedded sub-concepts. However, these structures are often incomplete and poorly-updated due to the effort of manually analyzing and re-analyzing learning materials.

Accordingly, embodiments described herein automatically analyze available learning materials to extract and organize the concepts discussed therein. Many learning materials inherently include structural information that may be helpful to identify concepts discussed in a particular learning material and relationships between discussed concepts. Thus, some embodiments described herein perform an analysis of the learning materials to extract the concepts and determine the relationships between the concepts. By extracting the concepts and the relationships therebetween, the concepts may be organized to allow learners to logically navigate through the concepts and that illustrates relationships between the concepts.

Throughout this application, the term “knowledge point” is used to refer to the “concepts” of the learning materials. The knowledge points of a learning material may include any term or set of words that represent a concept, a notion, an idea, etc. discussed or otherwise presented in the learning material. A knowledge point may include, for instance, the topics, the subtopics, and key terms of the learning materials. For example, a set of learning materials may pertain to a few courses on machine learning. The knowledge points may include topics and subtopics discussed in the courses such as neural networks, statistical inferences, clustering, and structural predictions.

Embodiments described herein generally identify, extract, and organize the knowledge points of learning materials. For example, some embodiments described herein may provide a learning support system which may automatically identify and extract fine-granularity knowledge points and relationships between the knowledge points from massive learning materials. These embodiments may further align the knowledge points with corresponding learning materials and provide links to the learning materials. By aligning the knowledge points with the corresponding learning materials, the learners may be provided with a specific resource that may provide additional information related to the knowledge point.

An example embodiment includes a method of automated domain knowledge structure generation. The domain knowledge structure may include a general organizational structure for the knowledge points. The method may include crawling learning materials and extracting the structural information from the learning materials. The method may also include extracting knowledge points from the learning materials. Hierarchy and dependency relationships between the knowledge points may be inferred and the knowledge points may be aligned with one or more of the learning materials. A domain knowledge structure may then be generated. The domain knowledge structure may include the extracted knowledge points organized at least partially according to the hierarchy and dependency relationships. The aligned learning materials may also be included in the domain knowledge structure. A learner may use the domain knowledge structure to gather information about one or more of the knowledge points and to find relationships between the one or more knowledge points and related knowledge points. Additionally, the learners may be pointed to the learning materials aligned with one or more of the knowledge points. This and other embodiments are described with reference to the appended drawings.

FIG. 1 illustrates a block diagram of an example personalized learning environment (learning environment) 100 in which some embodiments described herein may be implemented. The learning environment 100 may include an analysis server 108 that enables automated generation of a domain knowledge structure 140 from learning materials 130 and learning material metadata 132 . The domain knowledge structure 140 may generally include an organized representation of knowledge points extracted from the learning materials 130 . The analysis server 108 may be configured to extract the knowledge points and structural information from the learning materials 130 and metadata 132 . The analysis server 108 may be configured to generate the domain knowledge structure 140 . For example, the domain knowledge structure 140 may be based on the extracted knowledge points and the inferred hierarchy and dependency relationships among the knowledge points. After the domain knowledge structure 140 is generated, a learner 102 may be able to browse the domain knowledge structure 140 to learn a particular knowledge point that may be of interest.

The analysis server 108 may generate the domain knowledge structure 140 without or with minimal action by an individual. For example, manual operations such as reading, evaluating, and relating the learning materials 130 , which are generally performed by individuals may be included in operations performed by the analysis server 108 .

An example of the domain knowledge structure 140 may include a hierarchy of knowledge points. In the hierarchy, broad (e.g., more general) knowledge points may be included in higher levels of the hierarchy and narrow (e.g., more specific) knowledge points may be included in lower levels of the hierarchy. For example, a broad knowledge point such as “machine learning” may be included in a first level of the hierarchy and a narrow knowledge point such as “supervised learning” and “unsupervised learning” associated with machine learning may be included in a lower level of the hierarchy that is a sub-level of the first level. Thus, the learner 102 who may be interested in “supervised learning” may begin a navigation with a machine learning knowledge point and then may narrow her search to “supervised learning.” In other examples, the domain knowledge structure 140 may include an ontology, a cluster diagram, a list, an outline, or any other suitable organizational model.

Additionally, the analysis server 108 may align one or more of the learning materials 130 with a corresponding knowledge point in the domain knowledge structure 140 . The aligned learning materials may be linked or referenced in the domain knowledge structure to the corresponding knowledge point. For example, from the example above, a section of a document or portion of a video that includes information related to neural networks may be linked to the neural network knowledge point.

The learning environment 100 of FIG. 1 may include the analysis server 108 , a learning materials server 114 , and a learner device 104 . The learner device 104 , the learning materials server 114 , and the analysis server 108 may communicate via a network 122 . For example, the learner device 104 and the analysis server 108 may communicate the learning materials 130 and the learning material metadata 132 via the network 122 .

The network 122 may be wired or wireless, and may have numerous different configurations including a star configuration, a token ring configuration, or other configurations. Furthermore, the network 122 may include a local area network (LAN), a wide area network (WAN) (e.g., the Internet), and/or other interconnected data paths across which multiple devices may communicate. In some embodiments, the network 122 may include a peer-to-peer network. The network 122 may also be coupled to or include portions of a telecommunications network that may enable communication of data in a variety of different communication protocols.

In some embodiments, the network 122 includes BLUETOOTH® communication networks and/or cellular communications networks for sending and receiving data including via short messaging service (SMS), multimedia messaging service (MMS), hypertext transfer protocol (HTTP), direct data connection, wireless application protocol (WAP), e-mail, etc.

Communication via the network 122 may include actively transmitting data as well as actively accessing data. For example, in some embodiments, the learning materials server 114 may transmit the learning materials 130 to the analysis server 108 via the network 122 . Additionally, an analysis module 110 may crawl or otherwise access the learning materials 130 of the learning materials server 114 via the network 122 .

The learner 102 may include any individual or entity. In some embodiments, the learner 102 may be participating in an open learning course or may use the learning environment 100 for self-directed education. For example, the learner 102 may interface with the analysis server 108 with the intention of conducting research of a particular topic or for the purpose of learning about the particular topic. Accordingly, in these and other embodiments, the learner 102 may access the domain knowledge structure 140 rather than the learning materials 130 directly because the domain knowledge structure 140 may be better organized and/or more comprehensive than the learning materials 130 as stored on the learning materials server 114 .

The learner 102 may access the domain knowledge structure 140 via the learner device 104 . The learner device 104 may include a computing device that includes a processor, memory, and network communication capabilities. For example, the learner device 104 may include a laptop computer, a desktop computer, a tablet computer, a mobile telephone, a personal digital assistant (“PDA”), a mobile e-mail device, a portable game player, a portable music player, a television with one or more processors embedded therein or coupled thereto, or other electronic device capable of accessing the network 122 .

The learner device 104 may include a learner module 106 . In some embodiments, the learner module 106 may act in part as a thin-client application that may be stored on a computing device, such as the learner device 104 , and in part as components that may be stored on the analysis server 108 , for instance. In some embodiments, the learner module 106 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations), a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In some other instances, the learner module 106 may be implemented using a combination of hardware and software.

The learner module may enable interaction between the learner 102 and the analysis server 108 . For example, the learner module 106 may be configured to provide a user interface to a user interface device such as a user display 121 . The user interface may allow the learner 102 to access the domain knowledge structure 140 . In addition, the learner 102 may view the domain knowledge structure 140 or a portion thereof on the user display 121 . Additionally, the learner 102 may search for the domain knowledge structure 140 via the learner module 106 using the user interface device. For example, the user display 121 may include input capabilities or reflect user input received by another user input device.

The user display 121 may include any hardware device that receives data and information and generates a visual representation thereof. The user display 121 may additionally be configured to receive user input (e.g., configured as a touchscreen). Some examples of the user display 121 may include a cathode ray tube (CRT) display, a light-emitting diode (LED) display, an electroluminescent display, a liquid crystal display (LCD), or another suitable display.

In FIG. 1 , the learner device 104 is separate from the analysis server 108 and the learner module 106 included in the learner device 104 enables the learner 102 to access the analysis server 108 . In some embodiments, the learner 102 may interface directly with the analysis server 108 rather than using the learner module 106 . Additionally or alternatively, the learner device 104 may be used by the learner 102 to interface with the analysis server 108 via a browser. Additionally or alternatively, the learner module 106 may be configured to perform one or more operations attributed to the analysis module 110 . For example, the learner module 106 may generate the domain knowledge structure 140 based on the learning materials or some portion thereof. The domain knowledge structure 140 may be stored at the learner device 104 or another suitable storage location (e.g., cloud storage, a storage server, etc.).

The learning materials server 114 may include a hardware server that includes a processor, memory, and communication capabilities. In the illustrated embodiment, the learning materials server 114 may be coupled to the network 122 to send and receive data to and from the learner device 104 and the analysis server 108 via the network 122 . Additionally, the learning materials server 114 may be coupled to the network 122 such that the analysis module 110 may access the learning material metadata 132 and/or the learning materials 130 .

The learning materials server 114 may be configured to host and/or store the learning materials 130 and the learning material metadata 132 . The learning materials 130 may be organized according to a course to which the learning materials 130 pertain. The learning material metadata 132 may include metadata from the learning materials 130 . Some examples of the learning material metadata 132 may include a course title, a course number, a date or dates of the course, a professor, an institute, the syllabus, a title of one of the learning materials 130 such as the notes, and the text of the learning materials 130 .

The learning materials 130 may include academic course materials, syllabi, videos, example problems/solutions, lecture notes, lecture note slides, corresponding list, video transcripts, electronic books (e-books), seminars, and the like. The learning materials 130 may include or constitute open courseware (OCW), massive online open courses (MOOC), sparsely distributed learning materials such as course pages on professors' personal homepages, or any combination thereof.

The learning materials server 114 may be associated with an educational entity 138 . The educational entity 138 may upload or otherwise make available the learning materials 130 and the learning material metadata 132 . For example, the educational entity 138 may include a university and/or an education platform. Additionally, the educational entity 138 may include a professor or a department administrator.

For example, the educational entity 138 may include a university or another entity providing open education materials such as OCW. Examples may include the OCW provided by Massachusetts Institute of Technology (MIT) or Tokyo Institute of Technology (TIT). Moreover, the educational entity 138 may include an educational platform that hosts the MOOC such as Coursera, EdX, Udacity, and Futurelearn. Additionally, the educational entity 138 may include a professor or the department administrator that contributes a course webpage, which may be crawled and stored on the learning materials server 114 .

In some embodiments, the educational entity 138 may include an entity that is not directly associated with providing educational resources. For instance, the educational entity 138 may include publisher of e-books or a website that hosts videos or images. Examples of these types of educational entities 138 may include YOUTUBE®, an e-book distribution website, and online library, an online digital media store, etc.

The analysis server 108 may include a hardware server that includes a processor, a memory, and network communication capabilities. In the illustrated embodiment, the analysis server 108 may be coupled to the network 122 to send and receive data to and from the learner device 104 and/or the learning materials server 114 via the network 122 . The analysis server 108 may include the analysis module 110 . The analysis module 110 may be configured to analyze the learning materials 130 and the learning material metadata 132 . Additionally, the analysis module 110 may be configured to interact with the learner module 106 to analyze the learning materials 130 and the learning material metadata 132 and/or provide the domain knowledge structure 140 to the learner 102 .

In some embodiments, the analysis module 110 may be configured to generate the domain knowledge structure 140 . In some embodiments, the analysis module 110 may act in part as a thin-client application that may be stored on a computing device, such as the learner device 104 , and in part as components that may be stored on the analysis server 108 , for instance. In some embodiments, the analysis module 110 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations), a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In some other instances, the analysis module 110 may be implemented using a combination of hardware and software.

The analysis module 110 may generate the domain knowledge structure 140 with minimal or no manual actions taken by an individual. For example, in the learning environment 100 , the analysis module 110 may crawl the learning materials 130 . While crawling, the learning materials 130 , the analysis module 110 may identify, scan and copy content of the learning materials 130 and may extract the learning material metadata 132 .

The analysis module 110 may extract structural information by analyzing the learning materials 130 and the learning material metadata 132 . Generally, the structural information may indicate general organization and general content of the learning materials 130 . In some circumstances, the learning materials 130 may not be well organized. Additionally, in embodiments in which the learning materials 130 include a MOOC, the learning materials 130 may include a wide variety of information that may relate to multiple sparsely distributed courses. Nevertheless, the analysis module 110 may find structured or semi-structured information during crawling and may extract the structured or semi-structured information.

The analysis module 110 may then generate segmented term sequences from the extracted lists and the page-format analysis. The segmented term sequences may be bounded according to the detected sequence borders. The segmented term sequences may include positional information indicating a position in the learning materials 130 of one or more of the segmented term sequences.

The analysis module 110 may also extract knowledge points from the learning materials 130 and the learning material metadata 132 . In some embodiments, the analysis module 110 may derive candidate knowledge points from the segmented term sequences. For example, the analysis module may process the segmented term sequences to derive the knowledge points from the segmented term sequences. The analysis module 110 may also construct a hierarchy of the knowledge points.

Hierarchy relationships between the knowledge points may then be inferred. The hierarchy relationship between a first knowledge point and a second knowledge point may include a determination as to whether the first knowledge point is broader (e.g., having a coarser granularity), narrower (e.g., having a finer granularity), or includes a similar granularity to the second knowledge point. For example, the first knowledge point may include “supervised learning” and the second knowledge point may include “machine learning.” Thus, a hierarchy relationship between the first knowledge point and the second knowledge point may include the first knowledge point being narrower (or having a finer granularity) than the second knowledge point.

Dependency relationships between knowledge points may be inferred. The dependency relationship between a first knowledge point and a second knowledge point may include a determination whether the first knowledge point contributes or otherwise should be learned prior to the second knowledge point, the first knowledge point may be learned concurrently with the second knowledge point, or the second knowledge point contributes or otherwise should be learned prior to the first knowledge point. For example, the concept of logistic regression contributes to an understanding of neural networks. Accordingly, a dependency relationship may be determined between the concept neural networks and the concept of logical regression.

The analysis module 110 may then align one or more of the knowledge points with one or more of the learning materials 130 . The aligned learning materials may be based on granularity of a particular knowledge point. For example, a broad knowledge point may be linked to broader or larger learning material and a narrow knowledge point may be linked to a more distinct or smaller portion of the learning materials 130 . The analysis module 110 may then generate the domain knowledge structure 140 . The domain knowledge structure 140 may include the extracted knowledge points organized at least partially according to the extracted inferred hierarchy and dependency relationships and including the aligned learning materials.

Modifications, additions, or omissions may be made to the learning environment 100 without departing from the scope of the present disclosure. Specifically, embodiments of the learning environment 100 are depicted in FIG. 1 as including one learner 102 , one learning materials server 114 , one learner device 104 , and one analysis server 108 . However, the present disclosure applies to a learning environment 100 including one or more learners 102 , one or more learning materials servers 114 , one or more learner devices 104 , and one or more analysis servers 108 , or any combination thereof.

Moreover, the separation of various components in the embodiments described herein is not meant to indicate that the separation occurs in all embodiments. Additionally, it may be understood with the benefit of this disclosure that the described components may be integrated together in a single component or separated into multiple components.

In the learning environment 100 , memory such as memory in the learner device 104 , the analysis server 108 , and the learning materials server 114 may include a non-transitory memory that stores data for providing the functionality described herein. The memory may be included in storage that may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, flash memory, or some other memory devices. In some embodiments, the storage also includes a non-volatile memory or similar permanent storage device and media including a hard disk drive, a floppy disk drive, a CD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RW device, a flash memory device, or some other mass storage device for storing information on a more permanent basis.

FIG. 2 illustrates an example of the analysis server 108 including an example of the analysis module 110 . The analysis server 108 of FIG. 2 includes the analysis module 110 , a processor 224 , a memory 222 , a server display 225 , and a communication unit 226 . The components ( 110 , 222 , 224 , and 226 ) of the analysis server 108 may be communicatively coupled by a bus 220 .

The processor 224 may include an arithmetic logic unit (ALU), a microprocessor, a general-purpose controller, or some other processor array to perform computations and software program analysis. The processor 224 may be coupled to the bus 220 for communication with the other components (e.g., 110 , 222 , and 226 ). The processor 224 generally processes data signals and may include various computing architectures including a complex instruction set computer (CISC) architecture, a reduced instruction set computer (RISC) architecture, or an architecture implementing a combination of instruction sets. Although in FIG. 2 the analysis server 108 is depicted as including a single processor 224 , multiple processors may be included in the analysis server 108 . Other processors, operating systems, and physical configurations may be possible.

The memory 222 may be configured to store instructions and/or data that may be executed by the processor 224 . The memory 222 may be coupled to the bus 220 for communication with the other components. The instructions and/or data may include code for performing the techniques or methods described herein. The memory 222 may include a DRAM device, an SRAM device, flash memory, or some other memory device. In some embodiments, the memory 222 also includes a non-volatile memory or similar permanent storage device and media including a hard disk drive, a floppy disk drive, a CD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RW device, a flash memory device, or some other mass storage device for storing information on a more permanent basis.

The communication unit 226 may be configured to transmit and receive data to and from at least one of the learning materials server 114 and the learner device 104 , and the analysis server 108 depending upon where the analysis module 110 is stored. The communication unit 226 may be coupled to the bus 220 . In some embodiments, the communication unit 226 includes a port for direct physical connection to the network 122 or to another communication channel. For example, the communication unit 226 may include a USB, SD, CAT-5, or similar port for wired communication with the components of the learning environment 100 . In some embodiments, the communication unit 226 includes a wireless transceiver for exchanging data via communication channels using one or more wireless communication methods, including IEEE 802.11, IEEE 802.16, BLUETOOTH®, or another suitable wireless communication method.

In some embodiments, the communication unit 226 includes a wired port and a wireless transceiver. The communication unit 226 may also provide other conventional connections to the network 122 for distribution of files and/or media objects using standard network protocols including transmission control protocol/internet protocol (TCP/IP), HTTP, HTTP secure (HTTPS), and simple mail transfer protocol (SMTP), etc. In some embodiments, the communication unit 226 may include a cellular communications transceiver for sending and receiving data over a cellular communications network including via SMS, MMS, HTTP, direct data connection, WAP, e-mail, or another suitable type of electronic communication.

The server display 225 may include any hardware device that receives data and information from one or more of the processor 224 , the memory 222 , and the communication unit 226 via the bus 220 . Some examples of the server display 225 may include a CRT display, a LED display, an electroluminescent display, an LCD, or another suitable display. In some embodiments, the server display 225 may receive input from a learner (e.g., the learner 102 of FIG. 1 ) and communicate the input to one or more of the processor 224 , the memory 222 , and the communication unit 226 via the bus 220 .

In the illustrated embodiment of FIG. 2 , the analysis module 110 may include a crawl module 202 , a structural information extraction module 204 , a knowledge point extraction module 206 , a hierarchy module 228 , a dependency module 208 , and an alignment module 210 (collectively, the modules 240 ). Each of the modules 240 may be implemented as software including one or more routines configured to perform one or more operations. The modules 240 may include a set of instructions executable by the processor 224 to provide the functionality described herein. In some instances, the modules 240 may be stored in or at least temporarily loaded into the memory 222 of the analysis server 108 and may be accessible and executable by the processor 224 . One or more of the modules 240 may be adapted for cooperation and communication with the processor 224 and components of the analysis server 108 via the bus 220 .

The crawl module 202 may be configured to crawl the learning materials 130 and/or extract the learning material metadata 132 . For example, the crawl module 202 may perform operations performed by a web crawler, a web spider, an ant, an automatic indexer, a web scutter, or another suitable bot. The crawl module 202 may copy pages or some data included therein that the crawl module 202 visits and/or communicate information and data included in crawled learning materials 130 and/or extracted the learning material metadata 132 to the analysis module 110 .

In some embodiments, the learning materials 130 and/or the learning material metadata 132 or some portion thereof may be communicated to the analysis module 110 . For example, with reference to FIGS. 1 and 2 , the learning materials server 114 may communicate the learning materials 130 to the analysis server 108 . One or more of the other modules 240 may accordingly access data included in the learning materials 130 and/or the learning material metadata 132 . Additionally, in some embodiments, the one or more of the learning materials 130 and/or the learning material metadata 132 may be identified and/or extracted as described in U.S. application Ser. No. 13/732,036, entitled: “Specific Online Resource Identification And Extraction,” filed Dec. 31, 2012, which is incorporated herein by reference in its entirety.

The structural information extraction module 204 may be configured to extract structural information from the learning materials 130 and/or the learning material metadata 132 , which may indicate general organization and content of the learning materials 130 . For instance, the structural information extraction module 204 may extract lists of syllabi, lecture notes, and tables of contents, etc. included in the learning materials 130 . The structured information in the lists may relate to the granularity and general organization of the information included in the learning materials 130 . For example, a table of contents may include candidate knowledge points (e.g., the headings, subheadings, etc.) and may indicate which knowledge points have a similar granularity (e.g., each subheading may include a similar granularity).

For example, with reference to FIGS. 3A and 3B , an example syllabus 310 and an example lecture note list 336 are illustrated. The syllabus 310 and the lecture note list 336 may include examples of the learning materials 130 that include various structural or semi-structural information that the structural information extraction module 204 of FIG. 2 may extract. The syllabus 300 may include an institution indicator 302 , a course number 304 , a course title 306 , which may be examples of learning course metadata such as the learning course metadata 132 of FIG. 1 . In the syllabus 300 , the institution indicator 302 , the course number 304 , the course title 306 may indicate general information of a course to which the syllabus 310 belongs.

Additionally in the syllabus 310 , a first top level topic of machine learning course may include “supervised learning,” a second top level topic may include “learning theory,” and a third top level topic may include “unsupervised learning,” which are depicted by topic headings 308 , 312 , and 314 , respectively. Moreover, some sub-topics related to the first top level topic of supervised learning may include “support vector machines,” “model selection and feature selection,” ensemble methods: Bagging, boosting,” and “evaluation and debugging learning algorithms,” as depicted by sub-topics 316 that appear under the topic heading 308 . Likewise, sub-topics related to the second top level topic learning theory are depicted by sub-topics 350 that appear under the topic heading 312 .

Referring to FIG. 3B , in the lecture note list 336 , a list of lecture notes 334 may include lecture notes that relate to the sub-topics (e.g., 316 and 350 of FIG. 3A ) of a course on machine learning. From the list of lecture notes 334 and the syllabus 310 , the structural information extraction module 204 of FIG. 2 may ascertain the granularity of the sub-topics and generally organizational information. For example, the structural information extraction module 204 may ascertain that concepts of “support vector machines” “supervised learning,” “learning theory,” and “VC dimension” are concepts that are related to machine learning. Additionally, the structural information extraction module 204 may ascertain that “supervised learning” and “learning theory” are of a first granularity and that “support vector machines” and “VC dimension” are of a second granularity. Additionally still, the structural information extraction module 204 may ascertain that “support vector machines” is a sub-topic of “supervised learning” and that “VC dimension” is a sub-topic of “learning theory.”

Referring back to FIG. 2 , the structural information extraction module 204 may also conduct a page-format analysis of one or more pages of the learning materials 130 . The page-format analysis may examine the pages for indications of structural information in the learning materials 130 and/or the learning material metadata 132 .

For example, with reference to FIG. 3C , an example lecture slide 330 is illustrated. The lecture slide 330 may be an example of the learning materials 130 of FIGS. 1 and 2 . The lecture slide 330 may be analyzed by the structural information extraction module 204 of FIG. 2 . A title heading 328 of the lecture slide 330 “Prior Distribution” may be extracted. Additionally, a subheading 332 “Conjugate priors:” and/or a term 352 “posterior” included in the lower level may be extracted. Accordingly, the structural information extraction module 204 may ascertain that the lecture slide 330 generally relates to a concept of prior distribution as indicated by the heading 328 . Additionally, the structural information extraction module 204 may ascertain that a concept of “prior distribution” and “posterior” may be related to prior distribution. Moreover, the structural information extraction module 204 may ascertain that the concept of “prior distribution” may have a different granularity than “conjugate priors” and that the term “conjugate priors” may have a finer granularity than “prior distribution.”

Referring back to FIG. 2 , the structural information extraction module 204 may then generate segmented term sequences 212 . The segmented term sequences 212 may be based on the structural information in extracted lists and/or data acquired during page-format analysis. The segmented term sequences may be bounded according to the detected sequence borders, which may be detected by the structural information extraction module 204 . The sequence borders may be based on one or more of line breaks, table cell borders, sentence borders, and specific punctuation.

The description continues in the full USPTO document.