Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-21 rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract ideas without significantly more, making the invention ineligible subject matter.
Regarding claims 1, 11 and 21, step 1 says, “Is the claim to a process, machine, manufacture or composition of matter?” For claims 1, 11 and 21, the answer is yes because claims 1 and 21 recite a non-transitory computer-readable medium machine and a system respectively, which are both machines, and claim 11 recites a method, which is a process. Therefore, claims 1, 11 and 21 pass step 1.
For claim 1, it recites A non-transitory computer-readable medium comprising executable instructions, the executable instructions being executable by one or more processors to perform a method, the method comprising: A non-transitory computer-readable medium is a machine/manufacture and passes step 1.
For claim 11, it recites A method comprising: A method is a process and passes step 1.
For claim 21, it recites A system comprising at least one processor and at least one memory including executable instructions that when executed by the at least one processor cause the system to: A system is a machine and passes step 1.
Step 2A, prong 1 says, “Does the claim recite an abstract idea, law of nature, or nature phenomenon?” For claims 1, 11 and 21, the answer is yes, because the limitations are abstract ideas that a person having ordinary skill in the art could do using a mental process with the aid of pencil and paper. Therefore, claims 1, 11 and 21 fail step 2A, prong 1.
For claims 1, 11 and 21, they recite receiving 3D data of an interior of a building, the building having one or more stories and one or more rooms on the one or more stories; A person having ordinary skill in the art can mentally record the interior of a building and mentally receive the 3D data by drawing the interior on a blueprint.
classifying the 3D data by: A person having ordinary skill in the art can look at the 3D data and mentally classify it. Additionally, they can write details of the rooms on a blueprint as aid.
applying a trained model to classify the multiple 360 degree panoramic images; and A person having ordinary skill in the art can look at and mentally classify panoramic images to determine what type of images they are. Additionally, they could write down the classifications of panoramas on paper for notes as aid. Applying a trained model can be interpreted as an extra step using generic computer components. Training the model on the other hand, is not an abstract idea.
determining, based on applying the trained model to classify the multiple 360 degree panoramic images, one or more room classifications for the 3D data; A person having ordinary skill in the art can look at the details of the panoramas mentally classify the panoramic images and determine room classifications for the 3D data.
generating, based on the 3D data, one or more story identifications of the one or more stories and one or more room identifications of the one or more rooms; A person having ordinary skill in the art can mentally determine the room identifications and write the story classifications on blueprint(s) with the aid of pen and paper.
generating, based on the one or more story identifications and the one or more room identifications, a property layout of the building, the property layout including the one or more story identifications and the one or more room classifications; and A person having ordinary skill in the art can mentally determine the property layout and draw the property layout on a blueprint using pen and paper.
Step 2A, prong 2 says, “Does the claim recite additional elements that integrate the judicial exception into a practical application?” For claims 1, 11 and 21, the answer is no, because while they recite a trained model, applying a trained model is described at high level without any details and can be considered as generic computer component, and does not integrate the judicial exception into a practical application. Therefore, claims 1, 11 and 21 fail step 2A, prong 2.
For claim 21, it recites A system comprising at least one processor and at least one memory including executable instructions that when executed by the at least one processor cause the system to: A processor and a memory are additional elements but are generic computer components that cannot integrate abstract ideas into a practical application.
For claims 1, 11 and 21, they recite receiving multiple 360 degree panoramic images of the interior, the multiple 360 degree panoramic images associated with the 3D data; Receiving the panoramic images is an additional element but insignificant extra solution of mere data gathering.
applying a trained model to classify the multiple 360 degree panoramic images; and While applying a trained model can be interpreted as an additional element, it is described at high level without any details and can be considered as generic computer component, and does not integrate the judicial exception into a practical application.
providing the property layout for display. This is an insignificant extra step of output/display data.
Step 2B says, “Does the claim recite additional elements that amount to significantly more than the judicial exception?” For claims 1, 11 and 21, the answer is no, because while they recite a trained model, applying a trained model, as well as receiving 360 degree panoramic images and providing layouts for display are generic computer components and/or insignificant extra solutions and do not amount to significantly more than the judicial exception. Additionally, processors and memories as disclosed in claim 21, are additional elements that are generic computer components and do not amount to significantly more than the judicial exception. Therefore, claims 1, 11 and 21 fail step 2B.
Claims 1, 11 and 21 are ineligible subject matter because they do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 2 and 12, they recite The non-transitory computer-readable medium of claim 1 wherein applying the trained model to classify the multiple 360 degree panoramic images includes:
for each 360 degree panoramic image of the multiple 360 degree panoramic images:
dividing each 360 degree panoramic image into multiple sections; and A person having ordinary skill in the art could mentally divide each panoramic image into sections with the aid of pencil and paper.
applying the trained model to classify each section of the multiple sections, thereby obtaining multiple section classifications, A person having ordinary skill in the art could mentally classify different sections to determine what type of sections they are. Additionally, they could use pencil and paper for aid.
wherein determining, based on applying the trained model to classify the multiple 360 degree panoramic images, the one or more room classifications for the 3D data includes determining, based on the multiple section classifications for each 360 degree panoramic image, the one or more room classifications for the 3D data. A person having ordinary skill in the art could mentally identify rooms by looking at details of the sections/panoramic images.
Claims 2 and 12 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and the trained model as the additional element of generic computer component does not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 3 and 13, they recite The non-transitory computer-readable medium of claim 1 wherein generating, based on the 3D data, the one or more story identifications includes:
identifying walkable areas in the 3D data; A person having ordinary skill in the art could mentally identify walkable areas via walking to those areas.
clustering the walkable areas into one or more clusters for the one or more stories; A person having ordinary skill in the art could mentally classify the walkable areas into clusters by noting which story each walkable area is on with the aid of pencil and paper.
identifying, based on the one or more clusters, one or more floor surfaces; A person having ordinary skill in the art could mentally identify floor surfaces by looking at the patterns of the clusters.
identifying, for each floor surface of the one or more floor surfaces, one or more walls connected to each floor surface; and A person having ordinary skill in the art can determine in the mind which walls connect to each floor surface just by looking at where the floor surface or walls end.
generating, based on the one or more floor surfaces and the one or more walls connected to each floor surface of the one or more floor surfaces, the one or more story identifications. A person having ordinary skill in the art could mentally look at the patterns of floor surfaces and walls for each story and can identify the story based on what they observed with the aid of a blueprint.
Claims 3 and 13 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 4 and 14, they recite The non-transitory computer-readable medium of claim 3 wherein identifying the walkable areas in the 3D data includes:
generating, based on the 3D data, a 3D distance map; A person having ordinary skill in the art could mathematically generate a 3D distance map because distances are mathematical values.
determining, for each point of multiple points in the 3D distance map, a distance from each point to a nearest surface of the 3D data; and A person having ordinary skill in the art could mentally determine each point to a nearest surface and/or determining distances can be a mathematical concept of mathematical calculations.
identifying, based on the distance from each point to the nearest surface of the 3D data, the walkable areas in the 3D data. A person having ordinary skill in the art could determine in the mind a walkable area.
Claims 4 and 14 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 3 and 13 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 5 and 15, they recite The non-transitory computer-readable medium of claim 4 wherein identifying the walkable areas in the 3D data further includes:
generating an ellipsoid representing a human; A person having ordinary skill in the art could draw an ellipsoid using a pen and paper.
scaling down by a factor in a z-direction the ellipsoid to a sphere; and A person having ordinary skill in the art could use a mathematical calculation to scale down an ellipsoid to a sphere.
scaling down by the factor in the z-direction the 3D data, A person having ordinary skill in the art could use a mathematical calculation to scale down the 3D data.
wherein identifying, based on the distance from each point to the nearest surface of the 3D data, the walkable areas in the 3D data includes identifying, based on the distance from each point to the nearest surface of the 3D data and a diameter of the sphere, the walkable areas in the 3D data. A person having ordinary skill in the art could mentally measure the distance between a diameter of a sphere and the nearest surface or use a mathematical calculation.
Claims 5 and 15 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 4 and 14 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 6 and 16, they recite The non-transitory computer-readable medium of claim 1 wherein generating, based on the 3D data, the one or more room identifications includes:
determining, based on the 3D data, one or more potential room centers; A person having ordinary skill in the art could mathematically pinpoint a room center by measuring side lengths and angles.
determining one or more paths between one or more pairs of two potential room centers of the one or more potential room centers; A person having ordinary skill in the art could mentally trace a line between room centers on the blueprint.
determining, based on the one or more paths and the 3D data, one or more doorways; A person having ordinary skill in the art could look at the blueprint and mentally find the doorways.
blocking the one or more doorways to generate one or more blocked doorways; and A person having ordinary skill in the art could draw lines/symbols on the blueprint to indicate doorways as a mental process with simple tool of pen and paper.
generating, based on the one or more potential room centers and the one or more blocked doorways, the one or more room identifications. A person having ordinary skill in the art could mentally determine the rooms based on doorway and room center patterns.
Claims 6 and 16 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 7 and 17, they recite The non-transitory computer-readable medium of claim 1 wherein generating, based on the one or more story identifications and the one or more room identifications, the property layout of the building includes:
generating, based on the one or more story identifications, one or more graphs; A person having ordinary skill in the art could draw graphs with a pen and paper, or can generate a graph as a mathematical concept of mathematical relationship between the story identifications.
for each graph of the one or more graphs, simplifying each graph to generate a simplified graph, thereby generating one or more simplified graphs; and A person having ordinary skill in the art could draw a simplified version of a graph with a pen and paper, or can simplify the graph as a mathematical concept of mathematical relationship to remove edges and points in the graphs.
generating, based on the one or more simplified graphs and the one or more room identifications, the property layout of the building. A person having ordinary skill in the art could look at certain patterns on a graph to see where the rooms connect and draw a property layout based on those patterns as a mental process with simple tool such as pen and paper.
Claims 7 and 17 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 8 and 18, they recite The non-transitory computer-readable medium of claim 7 wherein simplifying each graph to generate the simplified graph includes simplifying each graph using at least one of a global nonlinear optimizer and one or more rule-based simplifications. A global nonlinear optimizer and one or more rule-based simplifications are mathematical concepts. Additionally, rule-based simplifications can be mental processes as well.
Claims 8 and 18 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 7 and 17 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 9 and 19, they recite The non-transitory computer-readable medium of claim 1, the method further comprising determining, for at least one room of the one or more rooms, at least one of a room area, a room volume, a first room dimension, and a second room dimension, wherein providing the property layout for display includes providing, for the at least one room of the one or more rooms, at least one of the room area, the room volume, the first room dimension, and the second room dimension for display. Determining room area, volume, dimensions can be mental process (a person can look at the rooms in the images/layout to estimate the room area, volume, dimensions), and/or a person can mathematically calculate the room area, volume, dimensions. Providing room area, volume or dimensions are additional elements but insignificant extra solutions for outputting the result.
Claims 9 and 19 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Regarding claims 10 and 20, they recite The non-transitory computer-readable medium of claim 1, the method further comprising:
receiving one or more modifications to the property layout; This is additional element but insignificant extra solution of mere data gathering.
generating, based on the one or more modifications, a modified property layout; and A person having ordinary skill in the art can add more details to the blueprint or create another blueprint, which can be a mental process with the simple tools such as pen and paper.
providing the modified property layout for display. This is an additional element of outputting results.
Claims 10 and 20 are ineligible subject matter because they only recite limitations that are abstract ideas that further limit claims 1 and 11 respectively and do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception and therefore, fail step 2.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3, 6-7, 9-11, 13, 16-17 and 19-21 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Khosravan, et al. (US 11830135 B1).
Regarding claims 1, 11 and 21, for claim 1, Khosravan teaches A non-transitory computer-readable medium comprising executable instructions, the executable instructions being executable by one or more processors to perform a method, the method comprising: (column 33, lines 34-45; “Thus, in some embodiments, some or all of the described techniques may be performed by hardware means that include one or more processors and/or memory and/or storage when configured by one or more software programs (e.g., by the BAMDM system 340 executing on server computing systems 300, by a Building Information Access system executing on server computing systems 300 or other computing systems/devices, etc.) and/or data structures, such as by execution of software instructions of the one or more software programs and/or by storage of such software instructions and/or data structures,”)
receiving 3D data of an interior of a building, the building having one or more stories and one or more rooms on the one or more stories; (column 2, lines 64-67, column 3, line 1 and column 6, lines 42-50; “such additional building information may, for example, include one or more of the following: a 3D, or three-dimensional, model of the building that includes height information (e.g., for building walls and other vertical areas);” and “one or more additional child vector embeddings at a next lower level of the hierarchy that are each associated with a parent higher-level vector embedding and represent parts of the area for the parent vector embedding (e.g., for a parent vector embedding that represents a building, to have one or more additional children vector embeddings that each represents a different floor or story or level of the building, or that groups rooms of the building in other manners, such as based on room types and/or sizes and/or functions;”)
classifying the 3D data by:
receiving multiple 360 degree panoramic images of the interior, the multiple 360 degree panoramic images associated with the 3D data; (column 3, lines 5-7; “images and/or other types of data captured in rooms of the building, including panoramic images (e.g., 360° panorama images); etc.,”)
applying a trained model to classify the multiple 360 degree panoramic images; and (column 28, lines 57-67 and column 29, lines 1-3; “Examples of node features that may be captured and used for the predictions including some or all of the following non-exclusive list: the number of doors, windows, and openings of the room; the room type; the perimeter of the room; the maximum length and width of room; the area of the room; the ratio between the room area and the room's bounding box area; chain code shape descriptors; shape descriptors to represent the centroid distance (e.g., the distance from the shape center to bound point); an order in which one or more images (e.g., panorama images) were captured in the room, relative to that of other rooms; features of the room extracted from analysis of one or more images (e.g., panorama images) captured in the room; center coordinates of the room; etc.”)
determining, based on applying the trained model to classify the multiple 360 degree panoramic images, one or more room classifications for the 3D data; (column 28, lines 32-33; “and at least one machine learning model trained to identify types of rooms.”)
generating, based on the 3D data, one or more story identifications of the one or more stories and one or more room identifications of the one or more rooms; (column 6, lines 42-50; “one or more additional child vector embeddings at a next lower level of the hierarchy that are each associated with a parent higher-level vector embedding and represent parts of the area for the parent vector embedding (e.g., for a parent vector embedding that represents a building, to have one or more additional children vector embeddings that each represents a different floor or story or level of the building, or that groups rooms of the building in other manners, such as based on room types and/or sizes and/or functions;”)
generating, based on the one or more story identifications and the one or more room identifications, a property layout of the building, the property layout including the one or more story identifications and the one or more room classifications; and (column 10, lines 38-41; “The automated operations may further include using the determined information to generate a floor plan for the building and to optionally generate other mapping information for the building,”)
providing the property layout for display. (column 19, lines 65-67 and column 20, lines 1-5 and fig 2D, component 260D; “FIG. 2D further illustrates one example 260d of a 2D floor plan for the house 198, such as may be presented to an end-user in a GUI, with the living room being the most westward room of the house (as reflected by directional indicator 209)—it will be appreciated that a 3D or 2.5D floor plan with rendered wall height information may be similarly generated and displayed in some embodiments, whether in addition to or instead of such a 2D floor plan.”)
For claim 11, it recites claim 1 in method and is rejected using the same rationale as claim 1.
For claim 21, Khosravan teaches A system comprising at least one processor and at least one memory including executable instructions that when executed by the at least one processor cause the system to: (column 33, lines 34-45; “Thus, in some embodiments, some or all of the described techniques may be performed by hardware means that include one or more processors and/or memory and/or storage when configured by one or more software programs (e.g., by the BAMDM system 340 executing on server computing systems 300, by a Building Information Access system executing on server computing systems 300 or other computing systems/devices, etc.) and/or data structures, such as by execution of software instructions of the one or more software programs and/or by storage of such software instructions and/or data structures,”)
The rest of claim 21 is a recitation of claim 1 and is rejected using the same rationale as claim 1.
Regarding claims 3 and 13, for claim 3, Khosravan teaches The non-transitory computer-readable medium of claim 1 wherein generating, based on the 3D data, the one or more story identifications includes:
identifying walkable areas in the 3D data; (column 23, lines 50-57 and fig 2F, components 243A-B; “Non-exclusive examples of attributes for a story-level node representing a story encompassing one or more rooms or other areas may include the following: attributes related to a layout of the encompassed room(s) and/or other area(s) (e.g., subjective attributes determined by one or more trained machine learning models), such as accessibility, walkability, visibility between multiple rooms and/or areas, etc.;”)
clustering the walkable areas into one or more clusters for the one or more stories; (column 23, lines 50-57 and fig 2F, components 243A-B; “Non-exclusive examples of attributes for a story-level node representing a story encompassing one or more rooms or other areas may include the following: attributes related to a layout of the encompassed room(s) and/or other area(s) (e.g., subjective attributes determined by one or more trained machine learning models), such as accessibility, walkability, visibility between multiple rooms and/or areas, etc.;”, the clusters would be the story-level nodes)
identifying, based on the one or more clusters, one or more floor surfaces; (column 23, lines 38-49 and fig 2F, components 210A-D, 243A-B, 244A-F, 251D-E; “Non-exclusive examples of attributes for an image-level node representing an image may include the following: per-pixel information for the visible element to which the pixel corresponds, such as depth, element type, etc.; features visible in the image (e.g., automatically identified by image analysis), such as objects, appliances, fixtures, surface materials (e.g., for floors, walls, countertops, appliances, etc.); a position at which the image was acquired, optionally within an enclosing room or other area; etc., and some or all such image-level attributes may further be included in a higher-level node in the graph representing a room or other area in which the image was acquired.”, floor material can be identified either at an image-level or room-level node, and room-level nodes are children of story-level nodes)
identifying, for each floor surface of the one or more floor surfaces, one or more walls connected to each floor surface; and (column 43, lines 14-27; “After block 637, the routine continues to block 640 to select the next room (beginning with the first) for which one or more images (e.g., 360° panorama images) acquired in the room are available, and to analyze the visual data of the image(s) for the room to determine a room shape (e.g., by determining at least wall locations), optionally along with determining uncertainty information about walls and/or other parts of the room shape, and optionally including identifying other wall and floor and ceiling elements (e.g., wall structural elements/features, such as windows, doorways and stairways and other inter-room wall openings and connecting passages, wall borders between a wall and another wall and/or ceiling and/or floor, etc.) and their positions within the determined room shape of the room.”)
generating, based on the one or more floor surfaces and the one or more walls connected to each floor surface of the one or more floor surfaces, the one or more story identifications. (column 44, lines 27-33; “Such a floor plan may include, for example, relative position and shape information for the various rooms without providing any actual dimension information for the individual rooms or building as a whole, and may further include multiple linked or associated sub-maps (e.g., to reflect different stories, levels, sections, etc.) of the building.”)
For claim 13, it recites claim 3 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 3.
Regarding claims 6 and 16, for claim 6, Khosravan teaches The non-transitory computer-readable medium of claim 1 wherein generating, based on the 3D data, the one or more room identifications includes:
determining, based on the 3D data, one or more potential room centers; (column 28, lines 57-67 and column 29, lines 1-3 and fig 2E, components 244A-L; “Examples of node features that may be captured and used for the predictions including some or all of the following non-exclusive list: the number of doors, windows, and openings of the room; the room type; the perimeter of the room; the maximum length and width of room; the area of the room; the ratio between the room area and the room's bounding box area; chain code shape descriptors; shape descriptors to represent the centroid distance (e.g., the distance from the shape center to bound point); an order in which one or more images (e.g., panorama images) were captured in the room, relative to that of other rooms; features of the room extracted from analysis of one or more images (e.g., panorama images) captured in the room; center coordinates of the room; etc.”, fig 2E, components 244A-L could potentially be room centers)
determining one or more paths between one or more pairs of two potential room centers of the one or more potential room centers; (fig 2E, component 235; discloses paths from one room center to another)
determining, based on the one or more paths and the 3D data, one or more doorways; (fig 2E, components 190, 194, 235 and 263; discloses determining doorways on the paths)
blocking the one or more doorways to generate one or more blocked doorways; and (fig 2E, components 190, 194, 235 and 263; discloses blocking doorways as seen on the floor plans and the adjacency graph)
generating, based on the one or more potential room centers and the one or more blocked doorways, the one or more room identifications. (column 28, lines 57-67 and column 29, lines 1-3, column 30, lines 30-42 and fig 2E, components 244A-L; “Examples of node features that may be captured and used for the predictions including some or all of the following non-exclusive list: the number of doors, windows, and openings of the room; the room type; the perimeter of the room; the maximum length and width of room; the area of the room; the ratio between the room area and the room's bounding box area; chain code shape descriptors; shape descriptors to represent the centroid distance (e.g., the distance from the shape center to bound point); an order in which one or more images (e.g., panorama images) were captured in the room, relative to that of other rooms; features of the room extracted from analysis of one or more images (e.g., panorama images) captured in the room; center coordinates of the room; etc.” and “the creation of an adjacency graph and/or associated vector embedding for a building may be further based in part on partial information that is provided for the building (e.g., by an operator user of the BAMDM system, by one or more end users, etc.). Such partial information may include, for example, one or more of the following: some or all room names for rooms of the building being provided, with the connections between the rooms to be automatically determined or otherwise established; some or all inter-room connections between rooms of the building being provided, with likely room names for the rooms to be automatically determined or otherwise established;”, fig 2E, components 244A-L could potentially be room centers)
For claim 16, it recites claim 6 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 6.
Regarding claims 7 and 17, Khosravan teaches The non-transitory computer-readable medium of claim 1 wherein generating, based on the one or more story identifications and the one or more room identifications, the property layout of the building includes:
generating, based on the one or more story identifications, one or more graphs; (column 23, lines 26-36 and fig 2F; “As discussed further with respect to FIG. 2F, an adjacency graph may in some embodiments include nodes at one or more additional levels of a hierarchy (e.g., a property level with each property-level node representing a property having one or more buildings and/or one or more external areas, a building level with each building-level node representing an entire building (optionally within a higher-level property), a story level with each story-level node representing a story or other level within a building and having one or more rooms or other areas, an image level with each image-level node representing a separate image, etc.),”)
for each graph of the one or more graphs, simplifying each graph to generate a simplified graph, thereby generating one or more simplified graphs; and (column 23, lines 26-34 and fig 2F; “As discussed further with respect to FIG. 2F, an adjacency graph may in some embodiments include nodes at one or more additional levels of a hierarchy (e.g., a property level with each property-level node representing a property having one or more buildings and/or one or more external areas, a building level with each building-level node representing an entire building (optionally within a higher-level property),”, the graph is simplified using a hierarchy where each level represents a story, room, etc.)
generating, based on the one or more simplified graphs and the one or more room identifications, the property layout of the building. (column 24, lines 42-49, fig 2F; “FIG. 2F further illustrates the use of a representation learning graph encoder component 265 of the BAMDM system that takes various hierarchical levels of information from the adjacency graph information as input, and that generates resulting hierarchical vector embeddings 275 that represent corresponding information in the adjacency graph 240e for the floor plan 230e.”)
For claim 17, it recites claim 7 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 7.
Regarding claims 9 and 19, for claim 9, Khosravan teaches The non-transitory computer-readable medium of claim 1, the method further comprising determining, for at least one room of the one or more rooms, at least one of a room area, a room volume, (column 8, lines 58-63 and column 15, lines 58-65; “such as the attributes and/or other characteristics that best enable subsequent automated identification of building floor plans having attributes satisfying target criteria (e.g., number of bedrooms; number of bathrooms; connectivity between rooms; size and/or dimensions of each room;” and “a floor plan (or portion of it) may be linked to or otherwise associated with one or more additional types of information, such as one or more associated and linked images or other associated and linked information, including for a two-dimensional (“2D”) floor plan of a building to be linked to or otherwise associated with a separate 2.5D model floor plan rendering of the building and/or a 3D model floor plan rendering of the building, etc.,”, size can be either the room area or room volume, depending on rather or not the floor plan is 2D or 3D) a first room dimension, and a second room dimension, (column 8, lines 58-63; “such as the attributes and/or other characteristics that best enable subsequent automated identification of building floor plans having attributes satisfying target criteria (e.g., number of bedrooms; number of bathrooms; connectivity between rooms; size and/or dimensions of each room;”) wherein providing the property layout for display includes providing, for the at least one room of the one or more rooms, at least one of the room area, the room volume, the first room dimension, and the second room dimension for display. (column 20, line 5-10 and fig 2D, component 260D; “Various types of information are illustrated on the 2D floor plan 260d in this example. For example, such types of information may include one or more of the following: room labels added to some or all rooms (e.g., “living room” for the living room); room dimensions added for some or all rooms;”, fig 2D, component 260D displays two dimensions as “20’ x 25’” in bedroom 1, and it would be inherent to add room area or volume because it is information)
For claim 19, it recites claim 9 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 9.
Regarding claims 10 and 20, for claim 10, Khosravan teaches The non-transitory computer-readable medium of claim 1, the method further comprising:
receiving one or more modifications to the property layout; (column 30, lines 30-42; “the creation of an adjacency graph and/or associated vector embedding for a building may be further based in part on partial information that is provided for the building (e.g., by an operator user of the BAMDM system, by one or more end users, etc.). Such partial information may include, for example, one or more of the following: some or all room names for rooms of the building being provided, with the connections between the rooms to be automatically determined or otherwise established; some or all inter-room connections between rooms of the building being provided, with likely room names for the rooms to be automatically determined or otherwise established;”, the partial information is the modification used to update a property layout)
generating, based on the one or more modifications, a modified property layout; and (column 28, lines 33-39; “The machine learning model(s) then provide output indicating predicted room types of some or all rooms in the building, with that information stored and optionally used to generate an updated version of the adjacency graph, and/or to otherwise update the information about the building (e.g., to update the building's floor plan, vector embedding(s), etc.).”)
providing the modified property layout for display. (column 19, lines 65-67 and column 20, lines 1-5 and fig 2D, component 260D; “FIG. 2D further illustrates one example 260d of a 2D floor plan for the house 198, such as may be presented to an end-user in a GUI, with the living room being the most westward room of the house (as reflected by directional indicator 209)—it will be appreciated that a 3D or 2.5D floor plan with rendered wall height information may be similarly generated and displayed in some embodiments, whether in addition to or instead of such a 2D floor plan.”)
For claim 20, it recites claim 10 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 10.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khosravan, et al. (US 11830135 B1) and Shen, et al. (Shen, Weichao "PanoViT: Vision Transformer for Room Layout Estimation from a Single Panoramic Image" 2022.).
Regarding claims 2 and 12, for claim 2, Khosravan teaches The non-transitory computer-readable medium of claim 1 wherein applying the trained model to classify the multiple 360 degree panoramic images includes:
for each 360 degree panoramic image of the multiple 360 degree panoramic images:
wherein determining, based on applying the trained model to classify the multiple 360 degree panoramic images, the one or more room classifications for the 3D data includes determining, (column 28, lines 57-67 and column 29, lines 1-3; “Examples of node features that may be captured and used for the predictions including some or all of the following non-exclusive list: the number of doors, windows, and openings of the room; the room type; the perimeter of the room; the maximum length and width of room; the area of the room; the ratio between the room area and the room's bounding box area; chain code shape descriptors; shape descriptors to represent the centroid distance (e.g., the distance from the shape center to bound point); an order in which one or more images (e.g., panorama images) were captured in the room, relative to that of other rooms; features of the room extracted from analysis of one or more images (e.g., panorama images) captured in the room; center coordinates of the room; etc.”)
However, Khosravan fails to explicitly teach dividing each 360 degree panoramic image into multiple sections; and
applying the trained model to classify each section of the multiple sections, thereby obtaining multiple section classifications,
the one or more room classifications for the 3D data includes determining, based on the multiple section classifications for each 360 degree panoramic image, the one or more room classifications for the 3D data.
Shen teaches dividing each 360 degree panoramic image into multiple sections; and (fig 4A-B; teaches dividing a panorama into sections known as patches)
applying the trained model to classify each section of the multiple sections, thereby obtaining multiple section classifications, (3.3 Vision transformer encoder; “The encoder first samples patches from the original image and the multi-scale feature maps. Then different patches are projected into different patch embeddings through different embedding operations. All the patch embeddings are attached with a recurrent position embedding to predict the room layout feature vector with a vision transformer backbone.”)
the one or more room classifications for the 3D data includes determining, based on the multiple section classifications for each 360 degree panoramic image, the one or more room classifications for the 3D data. (3.3 Vision transformer encoder; “The encoder first samples patches from the original image and the multi-scale feature maps. Then different patches are projected into different patch embeddings through different embedding operations. All the patch embeddings are attached with a recurrent position embedding to predict the room layout feature vector with a vision transformer backbone.”)
Before the effective filing date of the invention, it would be obvious for a person having ordinary skill in the art to add Khosravan’s non-transitory computer readable medium with Shen’s visual transformer because they both involve determining the layout of a room using a trained model, meaning that they are in the same field of endeavor. The person having ordinary skill in the art would be motivated to reduce inductive bias.
For claim 12, it recites claim 2 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 2.
Claim(s) 4 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khosravan, et al. (US 11830135 B1) in view of Chen (CN 110516564 A).
Regarding claims 4 and 14, for claim 4, Khosravan teaches The non-transitory computer-readable medium of claim 3 wherein identifying the walkable areas in the 3D data includes:
generating, based on the 3D data, a 3D distance map; (column 43, lines 31-40 and fig 6A, component 640; “while in other embodiments the room shape determination may be performed in other manners (e.g., by generating a 3D point cloud of some or all of the room walls and optionally the ceiling and/or floor, such as by analyzing at least visual data of the panorama image and optionally additional data captured by an image acquisition device or associated mobile computing device, optionally using one or more of SfM (Structure from Motion) or SLAM (Simultaneous Location And Mapping) or MVS (Multi-View Stereo) analysis).”, the point cloud will function as a 3D distance map)
the walkable areas in the 3D data. (column 23, lines 50-57 and fig 2F, components 243A-B; “Non-exclusive examples of attributes for a story-level node representing a story encompassing one or more rooms or other areas may include the following: attributes related to a layout of the encompassed room(s) and/or other area(s) (e.g., subjective attributes determined by one or more trained machine learning models), such as accessibility, walkability, visibility between multiple rooms and/or areas, etc.;”)
However, Khosravan fails to explicitly teach determining, for each point of multiple points in the 3D distance map, a distance from each point to a nearest surface of the 3D data; and
identifying, based on the distance from each point to the nearest surface of the 3D data, the walkable areas in the 3D data.
Chen teaches determining, for each point of multiple points in the 3D distance map, a distance from each point to a nearest surface of the 3D data; and (spec [0088]-[0090]; “The first detection subunit is used to detect the target point cloud data using a plane detection algorithm to detect a plane;” and “The second detection subunit is used to take point cloud data whose distance from the detected plane is less than a preset distance as the relevant point cloud data of the detected plane;”, the plane will function as the nearest surface and there is a distance check that would require checking each point to the nearest plane)
identifying, based on the distance from each point to the nearest surface of the 3D data, the walkable areas in the 3D data. (spec [0089]-[0090]; “The second detection subunit is used to take point cloud data whose distance from the detected plane is less than a preset distance as the relevant point cloud data of the detected plane;”, the relevant point cloud data of the detected plane will be the walkable area)
Before the effective filing date of the invention, it would be obvious for a person having ordinary skill in the art to add Chen’s plane detection unit to Khosravan’s non-transitory computer-readable medium to predictably detect walkable areas based on distance from each point to the nearest surface because they both disclose the use of a point cloud that functions as a 3D distance map for surfaces. The person having ordinary skill in the art would be motivated to check if a ceiling of an area would be too low for a person to walk to.
For claim 14, it recites claim 4 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 4.
Claim(s) 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khosravan, et al. (US 11830135 B1) in view of Chen (CN 110516564 A) as applied to claim(s) 4 and 14 above, and further in view of the Blender Documentation Team ("Blender 2.90 Manual" the Blender Documentation Team, 2020-2021.).
Regarding claims 5 and 15, for claim 5, Khosravan in view of Chen teaches The non-transitory computer-readable medium of claim 4 wherein identifying the walkable areas in the 3D data further includes:
wherein identifying, based on the distance from each point to the nearest surface of the 3D data, (Chen; spec [0084]; “the second detection sub-unit for the distance of the plane detected by less than the point of the predetermined distance cloud data as the related point cloud data of the plane detected;”) the walkable areas in the 3D data. (Khosravan; column 23, lines 50-57 and fig 2F, components 243A-B; “Non-exclusive examples of attributes for a story-level node representing a story encompassing one or more rooms or other areas may include the following: attributes related to a layout of the encompassed room(s) and/or other area(s) (e.g., subjective attributes determined by one or more trained machine learning models), such as accessibility, walkability, visibility between multiple rooms and/or areas, etc.;”)
Before the effective filing date of the invention, it would be obvious for a person having ordinary skill in the art to add Chen’s plane detection unit to Khosravan’s non-transitory computer-readable medium to predictably detect walkable areas based on distance from each point to the nearest surface because they both disclose the use of a point cloud that functions as a 3D distance map for surfaces. The person having ordinary skill in the art would be motivated to check if a ceiling of an area would be too low for a person to walk to.
However, Khosravan in view of Chen fails to explicitly teach generating an ellipsoid representing a human;
scaling down by a factor in a z-direction the ellipsoid to a sphere; and
scaling down by the factor in the z-direction the 3D data,
the walkable areas in the 3D data includes identifying, based on the distance from each point to the nearest surface of the 3D data and a diameter of the sphere, the walkable areas in the 3D data.
The Blender Documentation Team teaches generating an ellipsoid representing a human; (Structure, section Underlying Structure, Ellipsoid (ellipsoidal volume, tri-dimensional structure); “This is a meta which surface is generated by the field produced by an ellipsoidal volume. This gives an ellipsoidal surface.”)
scaling down by a factor in a z-direction (Transform, introduction, section Scale and fig titled “Transform Properties.”; “Use this panel to either edit or display the object’s transform properties such as position, rotation and/or scaling. These fields change the object’s origin and then affect the aspect of all its vertices and faces.” and “The object’s relative scale along the local axis (e.g. the Scale X value represents the scale along the local X axis). Each object (cube, sphere, etc.), when created, has a scale of one unit in each local direction. To make the object bigger or smaller, you scale it in the desired axis.”) the ellipsoid to a sphere; and (Structure, section Type, Underlying Structure, Ellipsoid (ellipsoidal volume, tri-dimensional structure); “The length, width and height of the ellipsoid.”, if an ellipsoid’s size z parameter was greater than the size x and y parameters, it would have more height than width or depth; lowering the size z parameter to equal the size x and y parameters would make it a sphere)
scaling down by the factor in the z-direction the 3D data, (Transform, introduction, section Scale and fig titled “Transform Properties.”; “Use this panel to either edit or display the object’s transform properties such as position, rotation and/or scaling. These fields change the object’s origin and then affect the aspect of all its vertices and faces.” and “The object’s relative scale along the local axis (e.g. the Scale X value represents the scale along the local X axis). Each object (cube, sphere, etc.), when created, has a scale of one unit in each local direction. To make the object bigger or smaller, you scale it in the desired axis.”, the z parameter of the 3D data can be changed to equal the z parameter of the ellipsoid)
the walkable areas in the 3D data includes identifying, based on the distance from each point to the nearest surface of the 3D data and a diameter of the sphere, (Structure, section Underlying Structure, Ellipsoid (ellipsoidal volume, tri-dimensional structure) and Measure, section Usage; “This is a meta which surface is generated by the field produced by an ellipsoidal volume. This gives an ellipsoidal surface.”, this surface will be the diameter and “Click and drag in the viewport to define the initial start/end point for the ruler.”, start/end points are the diameter and the nearest surface 3D data) the walkable areas in the 3D data. (Measure, fig captioned “Examples of the Measuring tool.”; the geometry shown on the figure will function as the walkable area)
Before the effective filing date of the invention, it would be obvious for a person having ordinary skill in the art to add Chen’s plane detection unit to Khosravan’s non-transitory computer-readable medium to predictably detect walkable areas based on distance from each point to the nearest surface because they both disclose the use of a point cloud that functions as a 3D distance map for surfaces. Then, the person having ordinary skill in the art can add Blender’s meta ellipsoid, scale cage and measuring tool to Khosravan’s non-transitory computer-readable medium combined with Chen’s plane detection unit to predictably yield a 3D model for checking walkable areas. The person having ordinary skill in the art would be motivated to better understand if a ceiling is too low.
For claim 15, it recites claim 5 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 5.
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khosravan, et al. (US 11830135 B1) in view of Yu, et al. (CN 111459166 A).
Regarding claims 8 and 18, for claim 8, Khosravan teaches The non-transitory computer-readable medium of claim 7 wherein simplifying each graph to generate the simplified graph includes simplifying each graph using one or more rule-based simplifications. (column 23, lines 26-34 and fig 2F, component 275; “As discussed further with respect to FIG. 2F, an adjacency graph may in some embodiments include nodes at one or more additional levels of a hierarchy (e.g., a property level with each property-level node representing a property having one or more buildings and/or one or more external areas, a building level with each building-level node representing an entire building (optionally within a higher-level property),”, the graph is simplified using a hierarchy where each level represents a story, room, etc., the rule being that a room-level node has to be a child of a story-level node and each image-level node must be the child of a room-level node, etc.)
However, Khosravan fails to explicitly teach at least one of a global nonlinear optimizer and
Yu teaches at least one of a global nonlinear optimizer and (spec [0049]; “Finally, before inserting the laser scan frame into the subgraph, the pose of the scan frame and the current subgraph need to be optimized using the Ceres Solver solver, which can transform the above problem into solving a nonlinear least squares problem.”, Ceres Solver is a global nonlinear optimizer)
Before the effective filing date of the invention, it would be obvious for a person having ordinary skill in the art to add Yu’s Ceres Solver to Khosravan’s non-transitory computer-readable medium to predictably yield optimization of a graph because both Khosravan and Yu uses graphs involving the layout of a building. The person having ordinary skill in the art would be motivated to optimize a graph for a better property layout.
For claim 18, it recites claim 8 in the form of a method and depends on claim 11 and is therefore rejected using the same rationale as claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IRVING SHI whose telephone number is (571)272-9613. The examiner can normally be reached Monday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IRVING NMN SHI/Examiner, Art Unit 2611
/HAIXIA DU/Primary Examiner, Art Unit 2611
Prosecution Insights