DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/forms/. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim(s) 1-20 is/are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of patent US 12229859. Although the claims at issue are not identical, they are not patentably distinct from each other because the pending claim(s) is/are an obvious variation of the patented claims, or entirely covered by the patented claims.
For example, claim 1 of the instant application discloses An apparatus, comprising: a display configured to receive touch input; a processor coupled to the display; and a memory that stores code executable by the processor to: display a two-dimensional representation of a floor plan; receive a user input associated with a portion of the floor plan; determine at least one three-dimensional chunk associated with the floor plan that intersects a line segment associated with the user input; gradually highlight a surface shown on the display that is associated with the at least one three-dimensional chunk, creating a highlighted area; and wherein the highlighted area gradually extends as a radius from a center point of the user input at a predefined rate. These limitations are all disclosed by the claim 1 of the patent US 12229859. Therefore, claim 1 of the instant application is covered by the claim 1 of the patent US 12229859, and is/are not patently distinct from the mentioned patent claim.
The following table illustrates a comparative mapping between the limitations of claim 1 of the instant application and the mapping claim 1 of patent US 12229859.
Claim 1 of the Instant Application 19020336
Claim 1 of the Patent 12229859
An apparatus, comprising:
a display configured to receive touch input;
a processor coupled to the display; and
a memory that stores code executable by the processor to:
An apparatus, comprising: a display configured to receive touch input; a processor coupled to the display; and a memory that stores code executable by the processor to:
display a two-dimensional representation of a floor plan;
receive a user input associated with a portion of the floor plan;
receive a user input associated with a surface of a structure presented on the display, the surface defining a portion of a floor plan;
generate, in response to the user input, a two-dimensional representation of the floor plan comprising a line segment representing the surface of the structure, the line segment defining a geometric plane in a third dimension;
determine at least one three-dimensional chunk associated with the floor plan that intersects a line segment associated with the user input;
determine at least one three-dimensional element associated with the line segment representing the surface on the two-dimensional representation of floor plan;
gradually highlight a surface shown on the display that is associated with the at least one three-dimensional chunk, creating a highlighted area; and
gradually highlight the surface shown on the display where the geometric plane that the line segment defines intersects with the at least one three-dimensional element associated with the line segment, the highlight presented as a graphical animation that extends from a center point of the user input to an ending point; and
wherein the highlighted area gradually extends as a radius from a center point of the user input at a predefined rate.
wherein the highlight gradually extends as a radius from the center point of the user input at a predefined rate.
The following is a complete listing of the correspondence between the claim of the instant application and the patents:
Claims of the Instant Application 19020336
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Claims of the Patent 12229859
1
2
3
4
5
6
7
8
9
10
11
12
14
13
15
16
2
17
19
20
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9, 16, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192).
Regarding claim 1, Jovanovic discloses An apparatus, comprising: a display configured to receive touch input (FIG. 1A, “[0021] The user may also directly interact with the user interface via touchscreen, keyboard, mouse key, touch pad and the like. [0023] The user devices 130, in some example embodiments, may include a Graphical User Interface (GUI) for displaying the user interface 140.”);
a processor coupled to the display; and a memory that stores code executable by the processor (“[0092] The example computer system 700 may be configured to include a processor or multiple processors 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory 704 and a static memory 706. [0093] Further, the disk drive unit 716 may include a computer-readable medium 722, on which is stored one or more sets of instructions and data structures (such as instructions 724) embodying or utilized by any one or more of the methodologies or functions described herein. Additionally, the instructions 724 may also reside, completely or partially, within the main memory 704 and/or within the processors 702 during execution by the computer system 700”) to:
display a two-dimensional representation of a floor plan (“[0053] FIG. 3A illustrates an example 2D environment 300. The example 2D environment 300 may include an interior space bound by a ground plane (e.g., a flooring surface) 340”);
receive a user input associated with a portion of the floor plan (“[0016] The user may select intersection points between different planes within the 2D environment. The different planes may include wall plane, ground plane (e.g., the flooring plane), top plane (e.g., the ceiling plane) and so forth. [0053] In the example shown here, the finger icon 350 or other suitable indicator may select a set of intersection points 312, 314, 316, 318, 320 on the ground.”);
determine at least one three-dimensional chunk associated with the floor plan that intersects a line segment associated with the user input (“[0020] FIGS. 3A, 3B and 3C are example representations of the 2D environment 300 with a user providing wall location on the 2D environment to form a 3D model of the 2D environment. [0053] The intersection points 312, 314, 316, 318, 320 may define points between wall planes 310 and ground plane 340. As the ground plane points are defined, the intersection points 313, 315, 317, 319 and 321 formed between the wall planes 310 and the top plane and intersection lines 330 between the ground plane points and corresponding intersection points are automatically generated.”).
On the other hand, Jovanovic fails to explicitly disclose but XING discloses gradually highlight a surface shown on the display, creating a highlighted area; and wherein the highlighted area gradually extends as a radius from a center point of the user input at a predefined rate (“[0267] Specifically, the electronic device may use the operating position of the image change operation as the center of a circle, gradually expand a circular display area from small to large on the first background image according to a preset radius increment, and display the image content of the corresponding position in the second background image within the circular display area until the circular display area completely covers the first background image. [0280] In the second background image, the second color is gradually changed to the third color from the center to the surroundings”. Therefore, the gradually expanded area corresponds to the highlight area).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Jovanovic and XING, to include all limitations of claim 1. That is, applying the displaying method of XING to a surface associated with the 3D model of Jovanovic. Therefore, gradually highlight a surface shown on the display that is associated with the at least one three-dimensional chunk. The motivation/ suggestion would have been the user can further customize the background image, which may further improve the user experience (XING, [0268]).
Regarding claim(s) 16, 19, they are interpreted and rejected for the same reasons set forth in claim(s) 1.
Regarding claim 9, Jovanovic in view of XING discloses The apparatus of claim 1.
On the other hand, Jovanovic fails to explicitly disclose but XING discloses wherein the highlighted area extends from the center point to an ending point and wherein the ending point comprises at least one of a corner or a point that is a threshold distance from the center point of the user input (“[0267] Specifically, the electronic device may use the operating position of the image change operation as the center of a circle, gradually expand a circular display area from small to large on the first background image according to a preset radius increment, and display the image content of the corresponding position in the second background image within the circular display area until the circular display area completely covers the first background image. [0280] In the second background image, the second color is gradually changed to the third color from the center to the surroundings”. Therefore, the gradually expanded area corresponds to the highlight area, and the distance from the center to the end point corresponds to the threshold distance). The same motivation of claim 1 applies here.
Claim(s) 2-4, 7, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192), and further in view of EDER et al. (US 20210279957).
Regarding claim 2, Jovanovic in view of XING discloses The apparatus of claim 1.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but EDER discloses wherein the code is executable by the processor to overlay a graphic representing a two-dimensional representation of floor plan on the display, creating an overlaid area (EDER, fig.13, “[0167] The final polygon POLY3 refers to the floor plan FLPN, which can be overlaid on the 3D model 5200 as shown in FIG. 13F. [0209] A reference object which has standard dimensions can be used to determine the dimensions of other items that are on the same plane as the reference item. One example to consider is if a user wants to estimate the floor plan/dimensions of a house and/or other portions of a location. The present system may be configured such that the user captures images of a room which has a floor made of tiles. A computer vision algorithm, such as mapping/3D reconstruction, that is processing these images can estimate the array of tiles in the scene and the user can then enter the size/dimensions of a single tile”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined EDER into the combination of Jovanovic and XING, to include all limitations of claim 2. That is, adding the floor plan overlay of EDER to the display of XING and Jovanovic. The motivation/ suggestion would have been providing a system that resolves several impediments in existing 3-dimensional (3D) visualization systems by creating a virtual representation of a location, and enabling this representation to be a platform for collaborative interaction for services and/or tasks to be performed by a user (EDER, [0006]).
Regarding claim(s) 17, it is interpreted and rejected for the same reasons set forth in claim(s) 2.
Regarding claim 3, Jovanovic in view of XING discloses The apparatus of claim 2, wherein receive a first user input associated with a first portion of the floor plan; determine at least a first three-dimensional chunk associated with the floor plan that intersects a line segment associated with the first user input; generate a first surface associated with the first three-dimensional chunk; and display the first surface, has/have been disclosed.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but EDER discloses “[0006] The system enables multiple users to synchronously or asynchronously utilize the virtual representation to collaboratively inspect, review, mark up, augment, and otherwise analyze the location entirely through one or more electronic devices (e.g., a computer, a phone, a tablet, etc.) in order to perform desired services and/or tasks at the location”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined EDER into the combination of Jovanovic and XING, to include all limitations of claim 3. That is, adding a second user of EDER to collaboratively utilize the display system with the first user of XING and Jovanovic. Therefore, receive a second user input associated with a second portion of the floor plan; determine at least a second three-dimensional chunk associated with the floor plan that intersects a line segment associated with the second user input; generate a second surface associated with the second three-dimensional chunk; and display the second surface. The same motivation of claim 2 applies here.
Regarding claim 4, Jovanovic in view of XING discloses The apparatus of claim 3, wherein determine at least one three-dimensional element representation of the floor plan; and gradually highlight the first surface shown on the display, the highlighted area presented as a graphical animation that extends from a center point of the first user input to a first ending point, has/have been disclosed.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but EDER discloses “[0006] The system enables multiple users to synchronously or asynchronously utilize the virtual representation to collaboratively inspect, review, mark up, augment, and otherwise analyze the location entirely through one or more electronic devices (e.g., a computer, a phone, a tablet, etc.) in order to perform desired services and/or tasks at the location”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined EDER into the combination of Jovanovic and XING, to include all limitations of claim 4. That is, adding a second user of EDER to collaboratively utilize the display system with the first user of XING and Jovanovic. Therefore, determine at least one three-dimensional element representation of the floor plan; and gradually highlight the second surface shown on the display, the highlighted area presented as a graphical animation that extends from a center point of the second user input to a second ending point. The same motivation of claim 2 applies here.
Regarding claim 7, Jovanovic in view of XING discloses The apparatus of claim 3.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but EDER discloses wherein the second surface comprises a different portion of the representation of the floor plan than the surface (“[0025] FIGS. 8A-8E illustrate an example construction (e.g., in real-time) of a virtual representation based on images of different portions of a room, according to an embodiment”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined EDER into the combination of Jovanovic and XING, to include all limitations of claim 7. That is, applying the different portions of a room of EDER to the two-dimensional representation of the floor plan of XING and Jovanovic. Therefore, the second surface comprises a different portion of the two-dimensional representation of the floor plan than the surface. The same motivation of claim 2 applies here.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192), and further in view of Montague (US 20130301953).
Regarding claim 8, Jovanovic in view of XING discloses The apparatus of claim 1.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but Montague discloses wherein the highlighted area comprises a semi- transparent color (Montague, “[0073] A number of different annotations 305 can be overlaid, such as but not limited to, an arrow 706 pointing to a feature, a dimension to indicate size, highlighted peanut shaped regions of a semi-transparent color, text, circles, squares, or other geometric shapes, or graphics, etc.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Montague into the combination of Jovanovic and XING. That is, applying the semi-transparent color of Montague to the replacing area of Jovanovic and XING, to include all limitations of claim 8. The motivation/ suggestion would have been to provide additional information (Montague, “[0134] a second heat map (not shown in figure) within viewer 500 and overlaid, in a semi-transparent manner, on top of subject object 100 can communicate additional viewer analytics information related to the currently displayed pose-angle 301, such as pointer position, pan, zoom, or measure characteristics”).
Claim(s) 10, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192), and further in view of Shantharam et al. (US 20190221040).
Regarding claim 10, Jovanovic in view of XING discloses The apparatus of claim 1.
On the other hand, Jovanovic in view of XING fails to explicitly disclose but Shantharam discloses wherein the user input comprises a touch input received on the display, the touch input graphically represented by a marker on the surface presented on the display (Shantharam, claim 14, “displaying, on the mobile device, a configuration mode for displaying the AR environment in real-time and obtaining user input data indicative of a position of a horizontal marker representing where an AR representation of a physical wall meets an AR representation of a physical floor in the AR environment, wherein the user input data includes touch screen input indicating the position where a user positioned the horizontal marker using a drag-and-drop gesture”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Shantharam into the combination of Jovanovic in view of XING. That is, applying the input represented by a marker of Shantharam to the user input of Jovanovic in view of XING, to include all limitations of claim 10. The motivation/ suggestion would have been The AR environment may further facilitate the selection, modification, and purchase of the simulated artwork (Shantharam, abstract).
Regarding claim(s) 20, it is interpreted and rejected for the same reasons set forth in claim(s) 10.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192), and further in view of Stout et al. (US 20230154110).
Regarding claim 12, Jovanovic in view of XING discloses The apparatus of claim 1, wherein
On the other hand, Jovanovic in view of XING fails to explicitly disclose but Stout discloses wherein the apparatus further comprises a light detection and ranging ("LiDAR") sensor that is used to generate a graphical mesh of a surface of a structure associated with at least one three-dimensional element representation of the floorplan, the graphical mesh overlaying an image of the structure (“[0060] The user then scans 110 the physical space, walking around scanning all walls, floors and ceilings with a visual sensor or camera utilizing AR technology, preferably a smartphone or computing device that employs LIDAR for most accurate results, and in some embodiments with a connected touch display, that may be part of the user/scanning computing device. As the user is scanning different surfaces and planes, SPP will distinguish between planes such as vertical walls, ceilings, and floors from objects such as cabinets, tables, plants and the like that are in the room. In one embodiment, SPP produces a geometric mesh polygon which is overlayed on the physical space that is scanned, and which can determine what is a plane versus what is an object.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Stout into the combination of Jovanovic in view of XING. That is, adding 3D mesh generated by LiDAR of Stout to surface of the structure of Jovanovic in view of XING, to include all limitations of claim 12. The motivation/ suggestion would have been to provide a method for creating a 3D virtual space based on physical locations and objects to facilitate virtual interactive experiences (Stout, [0002]).
Claim(s) 13, 14, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jovanovic (US 20180225393) in view of XING et al. (US 20220215192), and further in view of Stout et al. (US 20230154110) and EDER et al. (US 20210279957).
Regarding claim 13, Jovanovic in view of XING and Stout discloses The apparatus of claim 12.
Jovanovic further discloses wherein the two-dimensional representation of the floor plan comprises a line segment (FIG.3A, “[0016] A user may import photographic images, digital images, live video streams, and other graphical representations of a 2D environment. The user may select intersection points between different planes within the 2D environment. The user may further select or add points that automatically connect to previous points, thus providing lines of edges of a place”).
On the other hand, Jovanovic in view of XING and Stout fails to explicitly disclose but EDER discloses wherein the code is executable by the processor to highlight one or more vertices of at least one three-dimensional element that intersects a geometric plane that the line segment defines (EDER, fig.13F, “[0111] Based on the set of camera views through which each vertex can be seen and the set of subpixel locations to which the vertex corresponds, a color can be assigned to the vertex”). The same motivation of claim 2 applies here.
Regarding claim 14, Jovanovic in view of XING and Stout, EDER discloses The apparatus of claim 13.
On the other hand, Jovanovic in view of XING and EDER fails to explicitly disclose but Stout discloses wherein the at least one three-dimensional chunk is a three-dimensional chunk of the graphical mesh (“[0060] In one embodiment, SPP produces a geometric mesh polygon which is overlayed on the physical space that is scanned, and which can determine what is a plane versus what is an object. [0070] FIG. 8 presents the surveying and scanning process in progress. As can be seen mesh detection is used to cover the ground 810 as well as objects 820 in the physical space, while vertical walls 830 are identified separately by white virtual polygon shapes covering the plane”). The same motivation of claim 12 applies here.
Regarding claim 15, Jovanovic in view of XING and Stout, EDER discloses The apparatus of claim 14.
Jovanovic further discloses wherein the structure comprises a wall of a building (“[0020] FIGS. 3A, 3B and 3C are example representations of the 2D environment 300 with a user providing wall location on the 2D environment to form a 3D model of the 2D environment. FIG. 3D is an example of the 2D environment 300 along with a wall object”).
Allowable Subject Matter
Claim(s) 5-6, 11, 18 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 5, it recites, on the display, fade out the overlaid area of the graphic representing the two- dimensional representation of the floor plan and fade in a second overlaid area of a second graphic representing the second three-dimensional chunk. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 11, it recites, inter alia, wherein the apparatus further comprises a vertex shader that is used to determine edges of the at least one three-dimensional chunk that should be highlighted based on whether a geometric plane that the line segment defines intersects with the at least one three- dimensional chunk. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Regarding claim 18, it recites, inter alia, generating a second two-dimensional representation of the floor plan that is a copy of the two-dimensional representation of the floor plan and further includes a second line segment representing the second surface, the second line segment defining a second geometric plane in a third dimension; and replacing the two-dimensional representation of the floor plan with the second two- dimensional representation of the floor plan that includes the second line segment. None of the prior arts on the record or any of the prior arts searched, alone or in combination, renders obvious the combination of elements recited in the claim(s) as a whole.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRACE Q LI whose telephone number is (571)270-0497. The examiner can normally be reached Monday - Friday, 8:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DEVONA FAULK can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRACE Q LI/Primary Examiner, Art Unit 2618 9/26/2026
