DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Amendment
The office action is in response to Applicant’s amendment filed 06/09/2026 which has been entered and made of record. Claims 1-5 have been amended. Claims 9-20 have been newly added. Claims 1-20 are pending in the application. The claim interpretation under 35 U.S.C. 112(f) has been withdrawn based on the amendments of claim 1.
Response to Arguments
Applicant’s arguments, filed 06/09/2026, with respect to the rejection(s) under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of French and Wright as fully explained below.
Applicant argues French does not teach the newly added limitation of “wherein the knowledge graph comprises a plurality of distinct layers including a space layer with nodes describing spatial hierarchy, a stay event layer with nodes describing time-series stay information, an agent/object layer with nodes describing dynamic and static objects, and an activity layer with nodes describing actions”.
Examiner agrees. However, a new ground of rejection is made in view of French and Wright as fully explained below.
Applicant argues French does not disclose “matching a query graph against the knowledge graph to find a partial graph based on the semantic features”.
Examiner respectfully disagrees. First, in paragraph [0034] of the application’s specification, query graph is defined as a feature to be searched, “In the processing S1101, a query graph describing a feature in a scene to be searched for is received.”. Second, time feature is a type of semantic feature. Finally, French teaches a timeline specification as the semantic feature, and use it for a sub graph, in paragraph [0061] “A timeline 36 specification and preview window 39 permit the user to specify a time extent and viewpoint for a particular rendition of the scene graph 40, further teaches the clipping operation to restrict the part of graph based on the range of time in paragraph [0025] “clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”.
Conclusions: The rejections set in the previous Office Action are shown to have been proper, and the claims are rejected below. New citations and parenthetical remarks can be considered new grounds of rejection and such new grounds of rejection are necessitated by the Applicant's amendments to the claims. Therefore, the present Office Action is made final.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 10-11, 13 and 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over French et al. (US 20020032697 A1), hereinafter as French, in view of Wright et al. (US 20070171716 A1), hereinafter as Wright.
Regarding Claim 1, French teaches A scene recording and reconstructing device that records and reconstructs scenes in a virtual space (French paragraph [0018] “the present invention is a technique for representing a time varying visual scene as a directed acyclic graph of data and operators that generates a sequence of image frames over specified time intervals.”), the scene recording and reconstructing device comprising:
a geometry recording memory that records geometry information which describes a shape or an appearance of an object constituting a scene (French teaches a multimedia production system with storage as a geometry recording memory, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0019] “Elements of a scene are processed within the nodes of the graph. The nodes may process media data, such as images, video sequences, 3-D geometry, audio, or other data representative of the media elements.”, paragraph [0058] “The system 10 includes a number of media data object representations such as 3-D models 12, images 14, video 16, and audio 18, as well as the output from processes which operate on such data, such as image analysis 20 operations.”, paragraph [0066] “The scene graph 40 represents the internal state of the scene created by the user. Nodes in the graph 40 are operators 50 on multimedia data, or containers 52 for references to such multimedia data stored elsewhere in the system 10.”);
a record recording memory that records record information which is time-series information on an event performed by an object (French teaches a multimedia production system with storage as a record recording memory, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0019] “The nodes may also specify or modify control values or parameters for media elements. For example, the nodes may specify temporal and spatial values for the associated elements.”, paragraph [0065] “The parent-child relationship serves to define inherited characteristics. For example, the characteristics of a parent node are inherited by the children 46 and their descendants, but characteristics of the children do not affect a parent node. Such characteristics may include the general attributes such as color, or the temporal transformations described more fully herein below.”);
a knowledge graph recording memory that records a context of a scene as a knowledge graph (French teaches a multimedia production system with storage as a knowledge graph recording memory, a scene graph in Figure 1 40 and Figure 2 as the knowledge graph of the scene, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0018] “Each node in the graph represents an object oriented functional module that inherits a temporal as well as a spatial context, accepts and/or generates parameters, and processes some aspect of the scene. Directed data paths in the graph represent the flow of context, data (including media data) and/or control parameters as the graph is traversed from one node to another.”), ……
and a processor configured to reconstruct a desired scene in a virtual space, as required, by identifying a reconstruction target scene from a context of a scene recorded in the knowledge graph recording memory by matching a query graph against the knowledge graph to find a partial graph based on semantic features (French teaches a multimedia production system as a processor, and a timeline specification allowing user to choose a time range as the query graph, further teaches deciding a sub-graph for rendering based on the time range, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, Figure 1, paragraph [0061] “A timeline 36 specification and preview window 39 permit the user to specify a time extent and viewpoint for a particular rendition of the scene graph 40.”, paragraph [0025] “The branch nodes may apply temporal transformations of various types…… clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”),
acquiring geometry information and record information required to reconstruct the desired scene respectively from the geometry recording memory and the record recording memory, and re-combining the geometry information and the record information (French teaches “shape dino”, “shape stairs”, “render dino”, “render dino shadow” and “compositor” as nodes in scene graph, loading geometry model for “shape dino”, rendering geometry model, further teaches compositing multiple domains of output for viewer, paragraph [0074-0075] “The example scene graph 40 shown in FIG. 4 represents a demonstration scene of a computer animated dinosaur 60 walking into a live scene such as an office environment and which therefore must cast a shadow on a set of stairs 62. The scene graph 40 includes simplified networks of operators 50 including render passes, scenes, geometries and image analysis. The underlying regions are the domains for image compositing, 3D scenes, and shader spaces which generate texture images. As shown in FIG. 5A, operators such as the live scene 60 and the dinosaur model loader 61 are pushed down into macros 57. ”).
French is not relied on for the below claim language ……wherein the knowledge graph comprises a plurality of distinct layers including a space layer with nodes describing spatial hierarchy, a stay event layer with nodes describing time-series stay information, an agent/object layer with nodes describing dynamic and static objects, and an activity layer with nodes describing actions;…… Wright teaches ……wherein the knowledge graph comprises a plurality of distinct layers including a space layer with nodes describing spatial hierarchy, a stay event layer with nodes describing time-series stay information, an agent/object layer with nodes describing dynamic and static objects, and an activity layer with nodes describing actions;…… (Wright teaches using glyph, such as layers to represent data objects, further teaches Event Data Objects 20 as an activity layer, Entities data objects 24 as the agent/object layer; Location data objects 22 as the space layer and Event Data Objects 20 with time as the stay event layer, paragraph [0073] “Referring to FIG. 1, a tool information model is composed of the four basic data elements (objects 20, 22, 23, 24 and associations 26) that can have corresponding display elements in the visual representation 18. The four elements are used by the tool 12 to describe interconnected activities and information in time and space as the integrated visual representation 18”, paragraph [0079] “The Event is related to a location and a time at which the action took place, as well as several data properties and display properties including such as but not limited to; a short text label, description, location, start-time, end-time, general event type”, paragraph [0081] “Entities are data objects 24 that represent anything related to or involved in an event, including such as but not limited to; people, objects, organizations, equipment, businesses, observers”, paragraph [0082] “Locations are data objects 22 that represent a place within a spatial context/domain, such as a geospatial map, a node in a diagram such as a flowchart, or even a conceptual place”, paragraph [0112] “events are represented by a glyph, or icon as the visual element 410, placed along the timeline 422 at the point in time that the event occurred. The glyph can be actually a group of graphical objects, or layers”, paragraph [0129] “Locations are visual elements 410 represented by a glyph, or icon, placed on the reference surface 404 at the position specified by the coordinates in the corresponding location data object 22 (see FIG. 1). The glyph can be a group of graphical objects, or layers”, paragraph [0143] “The Entity glyph is actually a group of the entity visual elements 410 (e.g. graphical objects, or layers)”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Regarding Claim 2, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the knowledge graph is formed of nodes each corresponding either to individual geometry information or to individual record information, and a link describing a relationship between the nodes (French paragraph [0019] “Elements of a scene are processed within the nodes of the graph. The nodes may process media data, such as images, video sequences, 3-D geometry, audio, or other data representative of the media elements. The nodes may also specify or modify control values or parameters for media elements. For example, the nodes may specify temporal and spatial values for the associated elements.” And paragraph [0063] “as shown in FIG. 2, the scene graph 40 consists of a set of nodes 42 which are linked together in a special form of hierarchy, known as a directed acyclic graph (or "DAG"). The connections 44 between nodes 42 are directed, in the sense that they imply an asymmetric parent-child relationship. The graph is acyclic because the connections are not allowed to form a loop, which means child nodes cannot have links to their ancestors or themselves.”),
and the processor identifies a partial graph including a node corresponding to the geometry information, a node corresponding to the record information, and a link linking the nodes to each other (French teaches using a partial scene graph based on user’s input of timeline, paragraph [0262] “The time context for a traversal has a clip state, which is initialized from the shot extent. All elements of the scene are clipped to the shot extent. Time clip boxes accumulate (intersect) down the hierarchy as additional time clip operators are encountered during traversal. Clip state is pushed and popped on entering and leaving sub-graphs.”, paragraph [0024] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid.”),
and reconstructs a desired scene in a virtual space by reading the geometry information and the record information respectively from the geometry recording memory and the record recording memory and combining the geometry information and the record information (French teaches “shape dino”, “shape stairs”, “render dino”, “render dino shadow” and “compositor” as nodes in scene graph, loading geometry model for “shape dino”, rendering the geometry shape, further teaches compositing multiple domains of output for viewer, paragraph [0074-0075] “The example scene graph 40 shown in FIG. 4 represents a demonstration scene of a computer animated dinosaur 60 walking into a live scene such as an office environment and which therefore must cast a shadow on a set of stairs 62. The scene graph 40 includes simplified networks of operators 50 including render passes, scenes, geometries and image analysis. The underlying regions are the domains for image compositing, 3D scenes, and shader spaces which generate texture images. As shown in FIG. 5A, operators such as the live scene 60 and the dinosaur model loader 61 are pushed down into macros 57. ”).
Regarding Claim 3, French in view of Wright teaches The scene recording and reconstructing device according to claim 2, and further teaches wherein the processor expands the identified partial graph so as to include all objects and events having a same time range in a same space, and reconstructs a scene in a virtual space by using the expanded partial graph (French teaches propagation between sub-graphs through boundary/partition node, further teaches keeping the context to be the same, this implies an expanded sub-graph, paragraph [0244] “ it is useful to have a sub-graph boundary node which acts like a persistent save/restore operator for a single named object in the Catalog, but which can pass dependencies when required, during a single runtime session. When a Viewer is attached to one sub-graph within a large graph, data-changed events usually propagate throughout the graph, even if data is only recalculated for the operators actually traversed from the Viewer. The graph partition node could stop event propagation across the sub-graph boundary. If events are propagated, and traversal enabled, then the partition node does not modify any traversal contexts or data values.”).
Regarding Claim 4, French in view of Wright teach The scene recording and reconstructing device according to claim 2, and further teaches wherein the processor presents, as expansion candidates, an object and an event that are spatially or temporarily next or close to the identified partial graph, to a user, and allows the user to interactively select a scene reconstruction range based on user's selection/deletion of a candidate (French teaches displaying scene graph in user interface, further teaches user can manipulate timeline to exclude graph nodes in the graph, this implies user can interactively expand a partial graph based on adjusting the timeline on user interface to include or exclude a graph node, paragraph [0034] “The graph structure can be presented and/or manipulated in a user interface as a schematic diagram with nodes represented as shapes and connections between nodes as lines or arcs. Time transforms and time extents associated with graph nodes can also be presented and/or manipulated in a user interface as a time line with nodes represented as tracks and associated time transform and time extents represented as time intervals on these tracks.”, paragraph [0024-0025] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid…….clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”, paragraph [0219] “The scene graph can be presented and/or manipulated in a user interface as a schematic diagram, such as shown in FIG. 12 and 13. Time transforms and time extents can also be presented and/or manipulated in a user interface as tracks and time intervals in a time, such as shown in FIG. 14.”).
Regarding Claim 5, French teaches A scene recording and reconstructing method for a scene recording and reconstructing device configured to record and reconstruct scenes in a virtual space (French paragraph [0018] “the present invention is a technique for representing a time varying visual scene as a directed acyclic graph of data and operators that generates a sequence of image frames over specified time intervals.”), the scene recording and reconstructing method comprising:
recording geometry information which describes a shape or an appearance of an object constituting a scene (French paragraph [0019] “Elements of a scene are processed within the nodes of the graph. The nodes may process media data, such as images, video sequences, 3-D geometry, audio, or other data representative of the media elements.”, paragraph [0058] “The system 10 includes a number of media data object representations such as 3-D models 12, images 14, video 16, and audio 18, as well as the output from processes which operate on such data, such as image analysis 20 operations.”);
recording record information which is time-series information on an event performed by an object (French paragraph [0019] “The nodes may also specify or modify control values or parameters for media elements. For example, the nodes may specify temporal and spatial values for the associated elements.”, paragraph [0065] “The parent-child relationship serves to define inherited characteristics. For example, the characteristics of a parent node are inherited by the children 46 and their descendants, but characteristics of the children do not affect a parent node. Such characteristics may include the general attributes such as color, or the temporal transformations described more fully herein below.”);
recording a context of a scene as a knowledge graph (French teaches a scene graph in Figure 1 40 and Figure 2 as the knowledge graph of the scene, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0018] “Each node in the graph represents an object oriented functional module that inherits a temporal as well as a spatial context, accepts and/or generates parameters, and processes some aspect of the scene. Directed data paths in the graph represent the flow of context, data (including media data) and/or control parameters as the graph is traversed from one node to another.”), ……
and reconstructing a desired scene in a virtual space, as required, by identifying a reconstruction target scene from a recorded context of a scene by matching a query graph against the knowledge graph to find a partial graph based on semantic features, and by re-combining geometry information and record information necessary to reconstruct the desired scene (French teaches using a timeline specification allowing user to choose a sub-graph for rendering, “shape dino”, “shape stairs”, “render dino”, “render dino shadow” and “compositor” as nodes in scene graph, loading geometry model for “shape dino”, rendering geometry model, a timeline specification allowing user to choose a time range as the query graph, deciding a sub-graph for rendering based on the time range, further teaches compositing multiple domains of output for viewer, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, Figure 1, paragraph [0061] “A timeline 36 specification and preview window 39 permit the user to specify a time extent and viewpoint for a particular rendition of the scene graph 40.”, paragraph [0025] “The branch nodes may apply temporal transformations of various types…… clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”, paragraph [0074-0075] “The example scene graph 40 shown in FIG. 4 represents a demonstration scene of a computer animated dinosaur 60 walking into a live scene such as an office environment and which therefore must cast a shadow on a set of stairs 62. The scene graph 40 includes simplified networks of operators 50 including render passes, scenes, geometries and image analysis. The underlying regions are the domains for image compositing, 3D scenes, and shader spaces which generate texture images. As shown in FIG. 5A, operators such as the live scene 60 and the dinosaur model loader 61 are pushed down into macros 57. ”).
French is not relied on for the below claim language ……wherein the knowledge graph comprises a plurality of distinct layers including a space layer with nodes describing spatial hierarchy, a stay event layer with nodes describing time-series stay information, an agent/object layer with nodes describing dynamic and static objects, and an activity layer with nodes describing actions;…… Wright teaches ……wherein the knowledge graph comprises a plurality of distinct layers including a space layer with nodes describing spatial hierarchy, a stay event layer with nodes describing time-series stay information, an agent/object layer with nodes describing dynamic and static objects, and an activity layer with nodes describing actions;…… (Wright teaches using glyph, such as layers to represent data objects, further teaches Event Data Objects 20 as an activity layer, Entities data objects 24 as the agent/object layer; Location data objects 22 as the space layer and Event Data Objects 20 with time as the stay event layer, paragraph [0073] “Referring to FIG. 1, a tool information model is composed of the four basic data elements (objects 20, 22, 23, 24 and associations 26) that can have corresponding display elements in the visual representation 18. The four elements are used by the tool 12 to describe interconnected activities and information in time and space as the integrated visual representation 18”, paragraph [0079] “The Event is related to a location and a time at which the action took place, as well as several data properties and display properties including such as but not limited to; a short text label, description, location, start-time, end-time, general event type”, paragraph [0081] “Entities are data objects 24 that represent anything related to or involved in an event, including such as but not limited to; people, objects, organizations, equipment, businesses, observers”, paragraph [0082] “Locations are data objects 22 that represent a place within a spatial context/domain, such as a geospatial map, a node in a diagram such as a flowchart, or even a conceptual place”, paragraph [0112] “events are represented by a glyph, or icon as the visual element 410, placed along the timeline 422 at the point in time that the event occurred. The glyph can be actually a group of graphical objects, or layers”, paragraph [0129] “Locations are visual elements 410 represented by a glyph, or icon, placed on the reference surface 404 at the position specified by the coordinates in the corresponding location data object 22 (see FIG. 1). The glyph can be a group of graphical objects, or layers”, paragraph [0143] “The Entity glyph is actually a group of the entity visual elements 410 (e.g. graphical objects, or layers)”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Regarding Claim 6, French in view of Wright teach The scene recording and reconstructing method according to claim 5, and further teach wherein
the knowledge graph is formed of nodes each corresponding either to individual geometry information or to individual record information, and a link describing a relationship between the nodes (French paragraph [0019] “Elements of a scene are processed within the nodes of the graph. The nodes may process media data, such as images, video sequences, 3-D geometry, audio, or other data representative of the media elements. The nodes may also specify or modify control values or parameters for media elements. For example, the nodes may specify temporal and spatial values for the associated elements.” And paragraph [0063] “as shown in FIG. 2, the scene graph 40 consists of a set of nodes 42 which are linked together in a special form of hierarchy, known as a directed acyclic graph (or "DAG"). The connections 44 between nodes 42 are directed, in the sense that they imply an asymmetric parent-child relationship. The graph is acyclic because the connections are not allowed to form a loop, which means child nodes cannot have links to their ancestors or themselves.”),
and a partial graph formed of a node corresponding to the geometry information, a node corresponding to the record information, and a link linking the nodes to each other, is identified (French teaches using a partial scene graph based on user’s input of timeline, paragraph [0262] “The time context for a traversal has a clip state, which is initialized from the shot extent. All elements of the scene are clipped to the shot extent. Time clip boxes accumulate (intersect) down the hierarchy as additional time clip operators are encountered during traversal. Clip state is pushed and popped on entering and leaving sub-graphs.”, paragraph [0024] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid.”),
and the recorded geometry information and the recorded record information are combined, whereby a desired scene is reconstructed in a virtual space (French teaches “shape dino”, “shape stairs”, “render dino”, “render dino shadow” and “compositor” as nodes in scene graph, loading geometry model for “shape dino”, further teaches compositing multiple domains of output for viewer, paragraph [0074-0075] “The example scene graph 40 shown in FIG. 4 represents a demonstration scene of a computer animated dinosaur 60 walking into a live scene such as an office environment and which therefore must cast a shadow on a set of stairs 62. The scene graph 40 includes simplified networks of operators 50 including render passes, scenes, geometries and image analysis. The underlying regions are the domains for image compositing, 3D scenes, and shader spaces which generate texture images. As shown in FIG. 5A, operators such as the live scene 60 and the dinosaur model loader 61 are pushed down into macros 57. ”).
Regarding Claim 7, French in view of Wright teach The scene recording and reconstructing method according to claim 6, and further teach wherein the identified partial graph is expanded so as to include all objects and events having a same time range in a same space, and the expanded partial graph is used to reconstruct a scene in a virtual space (French teaches propagation between sub-graphs through boundary/partition node, further teaches keeping the context to be the same, this implies an expanded sub-graph, paragraph [0244] “ it is useful to have a sub-graph boundary node which acts like a persistent save/restore operator for a single named object in the Catalog, but which can pass dependencies when required, during a single runtime session. When a Viewer is attached to one sub-graph within a large graph, data-changed events usually propagate throughout the graph, even if data is only recalculated for the operators actually traversed from the Viewer. The graph partition node could stop event propagation across the sub-graph boundary. If events are propagated, and traversal enabled, then the partition node does not modify any traversal contexts or data values.”).
Regarding Claim 8, French in view of Wright teach The scene recording and reconstructing method according to claim 6, and further teach wherein as expansion candidates, an object and an event that are spatially or temporarily next or close to the identified partial graph are presented to a user, and the user is allowed to interactively select a scene reconstruction range based on user's selection/deletion of a candidate (French teaches displaying scene graph in user interface, further teaches user can manipulate timeline to exclude graph nodes, this implies user can interactively expand a partial graph based on adjusting the timeline on user interface to include or exclude a graph node, paragraph [0034] “The graph structure can be presented and/or manipulated in a user interface as a schematic diagram with nodes represented as shapes and connections between nodes as lines or arcs. Time transforms and time extents associated with graph nodes can also be presented and/or manipulated in a user interface as a time line with nodes represented as tracks and associated time transform and time extents represented as time intervals on these tracks.”, paragraph [0024-0025] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid……clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”, paragraph [0219] “The scene graph can be presented and/or manipulated in a user interface as a schematic diagram, such as shown in FIG. 12 and 13. Time transforms and time extents can also be presented and/or manipulated in a user interface as tracks and time intervals in a time, such as shown in FIG. 14.”).
Regarding claim 10, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein each node in the stay event layer stores a start time and an end time of a stay, a trajectory record indicating record information describing a trajectory during the stay, and a pose record indicating record information describing an action or gesture during the stay (Wright paragraph [0079] “The Event is related to a location and a time at which the action took place, as well as several data properties and display properties including such as but not limited to; a short text label, description, location, start-time, end-time, general event type”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Regarding claim 11, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the activity layer includes action nodes, and an action node of an action involving a doer and an objective person or object is linked to a node of the doer and a node of the objective person or object via respective relations (Wright teaches Entity X and Entity Y as the doer and objective person, further teaches the communication 1402 as the link between doer and objective person, Figure 14, paragraph [0195] “Referring to FIG. 14, example operation 1400 shows communications 1402 and movement events 1404 (connection visual elements 412-see FIGS. 6 and 7) between Entities "X" and "Y" over time on the visualization representation 18”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Regarding claim 13, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the processor is configured to add, to the partial graph, a stay event that is linked to a space element in the partial graph and that overlaps a candidate time range of the partial graph (French teaches displaying scene graph in user interface, further teaches user can manipulate timeline to exclude graph nodes in the graph, this implies user can interactively expand a partial graph based on adjusting the timeline on user interface to include or exclude a graph node, paragraph [0034] “The graph structure can be presented and/or manipulated in a user interface as a schematic diagram with nodes represented as shapes and connections between nodes as lines or arcs. Time transforms and time extents associated with graph nodes can also be presented and/or manipulated in a user interface as a time line with nodes represented as tracks and associated time transform and time extents represented as time intervals on these tracks.”, paragraph [0024-0025] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid…….clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”, paragraph [0219] “The scene graph can be presented and/or manipulated in a user interface as a schematic diagram, such as shown in FIG. 12 and 13. Time transforms and time extents can also be presented and/or manipulated in a user interface as tracks and time intervals in a time, such as shown in FIG. 14.”).
Regarding claim 15, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the agent/object layer includes nodes of dynamic objects, and a node of a dynamic object is linked to a node in the space layer via a stay node in the stay event layer to indicate a location of the dynamic object (Wright teaches an entity X as dynamic object, located at three different locations as nodes in space layer, and at different times as node in stay event layer in Figure 14, paragraph [0195] “This FIG. 14 shows a static view of Entity X making three phone call communications 1402 to Entity Y from 3 different locations 410a at three different times. Further, the movement events 1404 are shown on the visualization representation 18 indicating that the entity X was at three different locations 410a (location A,B,C), which each have associated timelines 422.”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Regarding claim 16, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the processor is configured to receive a query graph describing a feature in a scene to be searched for, and identify the partial graph that matches the query graph from scene context information stored in the knowledge graph recording memory (French teaches a timeline specification allowing user to choose a time range as the query graph, further teaches deciding a sub-graph for rendering based on the time range, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, Figure 1, paragraph [0061] “A timeline 36 specification and preview window 39 permit the user to specify a time extent and viewpoint for a particular rendition of the scene graph 40.”, paragraph [0025] “The branch nodes may apply temporal transformations of various types…… clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”),
Regarding claim 17, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the processor is configured to expand the partial graph so as to comprehensively include events having a same space and a same time, and perform automatic scene expansion of adjusting space and time ranges which are a target of scene reconstruction (French teaches propagation between sub-graphs through boundary/partition node, further teaches keeping the context to be the same, this implies an expanded sub-graph, paragraph [0244] “ it is useful to have a sub-graph boundary node which acts like a persistent save/restore operator for a single named object in the Catalog, but which can pass dependencies when required, during a single runtime session. When a Viewer is attached to one sub-graph within a large graph, data-changed events usually propagate throughout the graph, even if data is only recalculated for the operators actually traversed from the Viewer. The graph partition node could stop event propagation across the sub-graph boundary. If events are propagated, and traversal enabled, then the partition node does not modify any traversal contexts or data values.”).
Regarding claim 18, French in view of Wright teach The scene recording and reconstructing device according to claim 1, and further teach wherein the agent/object layer includes nodes of static objects, and a node of a static object is directly linked to a node in the space layer via a relation to indicate a location of the static object, and the node of the static object stores position coordinates of the static object (Wright teaches association between entity object and location object, paragraph [0108] “In the FIGS. 6 and 7 it is noted that event objects 20 are labeled as "Event 1", "Event 2", location objects 22 are labeled as "Location A", "Location B", and entity objects 24 are labeled as "Entity X", "Entity Y". The set of associations 16 are labeled as individual associations 26 with connections labeled as either solid or dotted lines 412 between two events, or dotted in the case of an indirect connection between two locations.”).
French and Wright are in the same field of endeavor, namely constructing virtual scene using graph structure. Wright teaches using multiple layers to represent data objects for knowledge graph in order to improve modeling efficiency (Wright paragraph [0091] “The visualization technique of the visualization tool 12 is designed to improve perception of entity activities, movements and relationships as they change over time in a concurrent time-geographic or time-diagrammatical context.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Wright with the method of French to improve modeling efficiency.
Claim(s) 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over French et al. (US 20020032697 A1), hereinafter as French.
Regarding claim 19, French teaches A scene recording and reconstructing apparatus that records and reconstructs a scene in a virtual space (French paragraph [0018] “the present invention is a technique for representing a time varying visual scene as a directed acyclic graph of data and operators that generates a sequence of image frames over specified time intervals.”), the apparatus comprising:
a geometry recording memory configured to record geometry information representing a shape or an appearance of an object constituting the scene (French teaches a multimedia production system with storage as a geometry recording memory, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0019] “Elements of a scene are processed within the nodes of the graph. The nodes may process media data, such as images, video sequences, 3-D geometry, audio, or other data representative of the media elements.”, paragraph [0058] “The system 10 includes a number of media data object representations such as 3-D models 12, images 14, video 16, and audio 18, as well as the output from processes which operate on such data, such as image analysis 20 operations.”, paragraph [0066] “The scene graph 40 represents the internal state of the scene created by the user. Nodes in the graph 40 are operators 50 on multimedia data, or containers 52 for references to such multimedia data stored elsewhere in the system 10.”);
a record recording memory configured to record information that is time-series information of an event performed by the object (French teaches a multimedia production system with storage as a record recording memory, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0019] “The nodes may also specify or modify control values or parameters for media elements. For example, the nodes may specify temporal and spatial values for the associated elements.”, paragraph [0065] “The parent-child relationship serves to define inherited characteristics. For example, the characteristics of a parent node are inherited by the children 46 and their descendants, but characteristics of the children do not affect a parent node. Such characteristics may include the general attributes such as color, or the temporal transformations described more fully herein below.”);
a knowledge graph recording memory configured to record a knowledge graph including a node corresponding to either individual geometry information or individual record information of the scene, and a link representing a context that is a relationship among the nodes (French teaches a multimedia production system with storage as a knowledge graph recording memory, a scene graph in Figure 1 40 and Figure 2 as the knowledge graph of the scene, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, paragraph [0018] “Each node in the graph represents an object oriented functional module that inherits a temporal as well as a spatial context, accepts and/or generates parameters, and processes some aspect of the scene. Directed data paths in the graph represent the flow of context, data (including media data) and/or control parameters as the graph is traversed from one node to another.”);
and a processor configured to reconstruct to: identify, in response to a request, a partial graph that is a scene to be reconstructed, the partial graph including a node corresponding to the geometry information recorded in the knowledge graph recording memory, a node corresponding to the record information, and a link representing a context that associates the nodes (French teaches a multimedia production system as a processor, and a timeline specification allowing user to choose a time range as a request, further teaches deciding a sub-graph for rendering based on the time range, paragraph [0038] “FIG. 1 is a block diagram of a multimedia production system which uses a scene graph according to the invention.”, Figure 1, paragraph [0061] “A timeline 36 specification and preview window 39 permit the user to specify a time extent and viewpoint for a particular rendition of the scene graph 40.”, paragraph [0025] “The branch nodes may apply temporal transformations of various types…… clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”);
expand the identified partial graph so that all objects and events within a same space and a same time range are included (French teaches propagation between sub-graphs through boundary/partition node, further teaches keeping the context to be the same, this implies an expanded sub-graph, paragraph [0244] “ it is useful to have a sub-graph boundary node which acts like a persistent save/restore operator for a single named object in the Catalog, but which can pass dependencies when required, during a single runtime session. When a Viewer is attached to one sub-graph within a large graph, data-changed events usually propagate throughout the graph, even if data is only recalculated for the operators actually traversed from the Viewer. The graph partition node could stop event propagation across the sub-graph boundary. If events are propagated, and traversal enabled, then the partition node does not modify any traversal contexts or data values.”);
and reconstruct a desired scene in the virtual space by reading the geometry information and the record information from the geometry recording memory and the record recording memory, respectively, and integrating the geometry information and the record information based on the expanded partial graph (French teaches “shape dino”, “shape stairs”, “render dino”, “render dino shadow” and “compositor” as nodes in scene graph, loading geometry model for “shape dino”, rendering geometry model, further teaches compositing multiple domains of output for viewer, paragraph [0074-0075] “The example scene graph 40 shown in FIG. 4 represents a demonstration scene of a computer animated dinosaur 60 walking into a live scene such as an office environment and which therefore must cast a shadow on a set of stairs 62. The scene graph 40 includes simplified networks of operators 50 including render passes, scenes, geometries and image analysis. The underlying regions are the domains for image compositing, 3D scenes, and shader spaces which generate texture images. As shown in FIG. 5A, operators such as the live scene 60 and the dinosaur model loader 61 are pushed down into macros 57. ”).
French and the current application are in the same field of endeavor, namely constructing virtual scene using graph structure. In various embodiments, French teaches using a scene graph with nodes, link and timeline to construct a virtual scene in order to improve modeling efficiency (French paragraph [0006] “This object-oriented scene model paradigm provides a number of important advantages. For example, model specifications, rather than becoming programs for rendering images and sounds, simply become descriptions of the objects in the scene and their properties and dynamic behaviors. These types of systems can be used to construct models in a natural way because the end-users can think in terms of abstract or real world objects, and therefore need not have the expertise nor even the interest in traditional graphics or real-time programming. Such models also tend to be more robust since they do not tend to exhibit side effects that interfere in subtle ways with the effects of other components, while providing other advantages such as economies of scale, usefulness, and longevity as well as automatic level of detail management.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of various embodiments of French to improve modeling efficiency.
Regarding claim 20, French teaches The scene recording and reconstructing apparatus according to claim 19, and further teach wherein the processor is configured to present, as expansion candidates, an object and an event that are spatially or temporally adjacent or proximate to the identified partial graph to a user, and to allow the user to interactively select a reconstruction range of the scene based on selection or exclusion of the candidates by the user (French teaches displaying scene graph in user interface, further teaches user can manipulate timeline to exclude graph nodes in the graph, this implies user can interactively expand a partial graph based on adjusting the timeline on user interface to include or exclude a graph node, paragraph [0034] “The graph structure can be presented and/or manipulated in a user interface as a schematic diagram with nodes represented as shapes and connections between nodes as lines or arcs. Time transforms and time extents associated with graph nodes can also be presented and/or manipulated in a user interface as a time line with nodes represented as tracks and associated time transform and time extents represented as time intervals on these tracks.”, paragraph [0024-0025] “Any temporal clipping results in excluding the node from contributing to the scene based upon an allowed range of time values for which the node is valid…….clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph outside a specified time window”, paragraph [0219] “The scene graph can be presented and/or manipulated in a user interface as a schematic diagram, such as shown in FIG. 12 and 13. Time transforms and time extents can also be presented and/or manipulated in a user interface as tracks and time intervals in a time, such as shown in FIG. 14.”).
Allowable Subject Matter
Claims 9, 12 and 14 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 9, the closest prior art of Wright teaches location data object (Wright paragraph [0082] “Locations are data objects 22 that represent a place within a spatial context/domain, such as a geospatial map, a node in a diagram such as a flowchart, or even a conceptual place such as "Shang-ri-la" or other "locations" that cannot be placed at a specific physical location on a map or other spatial domain. Each Location data object 22 can store such as but not limited to; position coordinates, a label, description, color information, precision information, location type, non-geospatial flag and user comments”). However, Wright fails to teach the combined limitation below as a whole, “wherein the space layer includes a district node, a building node, a floor node, and a room node, and the nodes in the space layer are linked by relations indicating inclusion relations and positional relations”. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claim 9 is considered to allowable.
Regarding claim 12, the closest prior art of French teaches propagation between sub-graphs through boundary/partition node, further teaches keeping the context to be the same. However French fails to teach the combined limitation below as a whole “when a plurality of space elements that are not adjacent to each other are included in the partial graph, identify a space element which is a common parent of the space elements on the knowledge graph and add the common parent to the partial graph.”. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claim 12 is considered to allowable.
Regarding claim 14, the closest prior art of French teaches propagation between sub-graphs through boundary/partition node. However French fails to teach the combined limitation below as a whole “wherein the processor is configured to identify, for a space element included in the partial graph, a parent space element thereof and a space element spatially adjacent thereto on the knowledge graph, and present the identified space elements as expansion candidates to a user”. Furthermore, no prior art of record either alone or in combination teaches the above limitation as a whole. Therefore, claim 14 is considered to allowable.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/XIAOMING WEI/ Examiner, Art Unit 2611
Prosecution Insights