Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/24/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4-13 and 16-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
On January 7, 2019, the USPTO released new examination guidelines setting forth a two-step inquiry for determining whether a claim is directed to non-statutory subject matter. According to the guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1 and 13 are directed toward non-statutory subject matter, as shown below:
STEP 1: Do claims 1 and 13 fall within one of the statutory categories? Claims 1 and 13 are a method and an apparatus respectively, and as such fall within one of the statutory categories.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? Yes, claims 1 and 13 are directed to mental processes.
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
NOTE: Claims 1 and 13 limitations that are similar in scope, hence examiner will assess method of claim 1.
The method of claim 1 contains a mental process that can be practicably performed in the human mind and, therefore, an abstract idea. The claim merely recites comparing features of a room derived from modified images captured by mobile device and images captured by mobile robot, identify a match between features and determine a mapping between point of view of mobile device camera and point of view of mobile robot camera. A human can perform all these mentally or with the aid of pen and paper.
Therefore, limitation following limitations are directed towards an abstract idea, “comparing features of a room derived from the modified images captured by the camera of the mobile device with features of the room derived from the images captured by the camera of the mobile robot;
identifying a match between the features derived from the modified images captured by the camera of the mobile device and the features derived from the images captured by the camera of the mobile robot;
determining a mapping between a first coordinate system associated with the point-of-view of the camera of the mobile device and a second coordinate system associated with a point-of-view of the camera of the mobile robot based at least on the identified match”
Furthermore, the following limitation merely displays graphic elements for user to initiate navigation, “causing a user interface of the mobile device to display one or more graphic elements that enable a user of the mobile device to initiate navigation of the mobile robot to a user-specified location based at least on the determined mapping”
Initiating the navigation does not necessitate executing the navigation. Thinking about how and where the mobile robot should navigate to is initiating navigation. And displaying the graphic elements is observation, evaluation, judgement and opinion. See Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016)
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? No, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claims 1 and 13 not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
The steps of, “applying an image transformation to images captured by a camera of a mobile device to generate modified images that simulate images captured by a camera of a mobile robot from a point-of-view of the camera of the mobile device, wherein the image transformation takes into account characteristics of the camera of the mobile device and characteristics of the camera of the mobile robot”, is merely apply it to convert images to assist in the abstract idea. Therefore, is not considered a practical application or significantly more.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claims 1 and 13 not recite any specific limitation or combination of limitations that are not well-understood, routine, conventional (WURC) activity in the field. Limitations identified as “apply it” in step 2A qualify as apply it in step 2B as well.
CONCLUSION
Thus, since claims 1 and 13 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that claim 1 and 13 are directed towards non-statutory subject matter.
With respect to claims 2 and 14, “instruct the mobile robot to perform a cleaning operation” is not considered a mental process and necessitated robot to perform a cleaning operation. Therefore, the claims are statutory.
Claims 3 and 15 are statutory as being dependent on claims 2 and 14 respectively.
With respect to claims 4 and 16, the claim further limits how the image transformation is performed. This is part of apply it and it not considered a practical application or significantly more.
With respect to claims 5 and 17, deriving illumination invariant features from images is performing image analysis. This is apply it and is not considered a practical application or significantly more.
With respect to claims 6 and 18, “receiving data associated with an environment of the mobile robot” is mere data gathering, hence an insignificant extra solution activity. And “overlaying the one or more graphic elements on the user interface of the mobile device” is collecting information, analyzing it, and displaying certain results of the collection and analysis, hence and abstract idea. Therefore, is not considered a practical application or significantly more.
With respect to claims 7 and 19, the claim further limits what graphic elements represent. This is still part of the abstract idea and not considered a practical application or significantly more.
With respect to claims 8 and 20, determining position of obstacle, portions occluded by obstacle, and displaying representation of environment occluded by obstacle is still collecting information, analyzing it, and displaying certain results of the collection and analysis, hence and abstract idea. Therefore, is not considered a practical application or significantly more.
With respect to claim 9, converting coordinates into real world coordinates is a mathematical calculation. Therefore, is still part of the abstract idea and not considered a practical application or significantly more.
With respect to claim 10, the claim merely defines floor surface that is to be cleaned. This does integrate to the method or apparatus to actually control the mobile robot. Therefore, is not considered a practical application or significantly more.
With respect to claim 11, executing an augmented reality session is still part of displaying information, hence is considered an abstract idea.
With respect to claim 12, establishing a coordinate system based on captured images is till part of displaying information, hence an abstract idea and is not considered a practical application or significantly more.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-19 of U.S. Patent No. 11,669,086. Although the claims at issue are not identical, they are not patentably distinct from each other because:
Claim of Instant Application 19193159
Claim of US Patent 11669086
1. A method comprising:
applying an image transformation to images captured by a camera of a mobile device to generate modified images that simulate images captured by a camera of a mobile robot from a point-of-view of the camera of the mobile device, wherein the image transformation takes into account characteristics of the camera of the mobile device and characteristics of the camera of the mobile robot;
comparing features of a room derived from the modified images captured by the camera of the mobile device with features of the room derived from the images captured by the camera of the mobile robot;
identifying a match between the features derived from the modified images captured by the camera of the mobile device and the features derived from the images captured by the camera of the mobile robot;
determining a mapping between a first coordinate system associated with the point-of-view of the camera of the mobile device and a second coordinate system associated with a point-of-view of the camera of the mobile robot based at least on the identified match; and
causing a user interface of the mobile device to display one or more graphic elements that enable a user of the mobile device to initiate navigation of the mobile robot to a user-specified location based at least on the determined mapping.
A method comprising:
determining, using at least one processor, a mapping between a first coordinate system associated with a point-of-view of a camera of a mobile device and a second coordinate system associated with a point-of-view of a camera of a mobile robot, in which the first coordinate system is different from the second coordinate system, the determining comprising:
applying an image transformation to images captured by the camera of the mobile device to generate modified images, wherein the modified images simulate images captured by the camera of the mobile robot from the point-of-view of the camera of the mobile device, and wherein the image transformation takes into account characteristics of the mobile device camera and characteristics of the mobile robot camera, comparing features of a room derived from the images captured by the camera of the mobile device with features of the room derived from the images captured by the camera of the mobile robot, wherein a floor surface of the room is to be cleaned by the mobile robot during a cleaning operation, and wherein the comparing comprises comparing features of the room derived from the modified images with the features of the room derived from the images captured by the camera of the mobile robot, identifying a match between a first feature derived from one of the images captured by the camera of the mobile device and a second feature derived from one of the images captured by the camera of the mobile robot, wherein the identifying comprises identifying a match between a first set of features derived from the modified images and a second set of features derived from the images captured by the camera of the mobile robot, identifying first coordinates associated with the first feature and second coordinates associated with the second feature, in which the first coordinates are based on the first coordinate system and the second coordinates are based on the second coordinate system, and
determining the mapping between the first coordinate system and the second coordinate system based on a relationship between the first coordinates and the second coordinates; and
providing at the mobile device a user interface to enable a user to interact with the mobile robot by using the mobile device to specify a location to which the mobile robot is navigated, wherein navigating the mobile robot to the specified location involves usage of the mapping between the first coordinate system and the second coordinate system.
2. The method of claim 1, wherein the user interface is further configured to enable the user to instruct the mobile robot to perform a cleaning operation at the user- specified location.
The method of claim 1 in which the user interface is configured to enable the user to use the mobile device to instruct the mobile robot to perform a specified action at the specified location
3. The method of claim 2, further comprising:
identifying first coordinates of the user-specified location in the first coordinate system;
converting the first coordinates to second coordinates in the second coordinate system using the determined mapping; and
transmitting, from the mobile device to the mobile robot, an instruction indicative of the second coordinates, wherein the instruction is configured to cause the mobile robot to perform the cleaning operation at the user-specified location.
The method of claim 2, comprising:
configuring the user interface to enable the user to specify the location on an image shown on a display module of the mobile device, identifying, using at least one processor, coordinates of the location specified by the user, and sending an instruction from the mobile device to the mobile robot to instruct the mobile robot to perform the cleaning operation at the location specified by the user, in which the instruction includes the coordinates of the location specified by the user.
4. The method of claim 3, comprising:
determining first coordinates of the location specified by the user, in which the first coordinates are based on the first coordinate system, converting the first coordinates to second coordinates using the mapping between the first coordinate system and the second coordinate system, in which the second coordinates are based on the second coordinate system, and
4. The method of claim 1, wherein applying the image transformation comprises applying the image transformation according to at least one of a sensor resolution, a lens focal length, a sensor size, a lens distortion, a lens field of view, or a sensor gain of the camera of the mobile device or the mobile robot.
1…wherein the image transformation takes into account characteristics of the mobile device camera and characteristics of the mobile robot camera
5. The method of claim 1, wherein the features of the room comprise a first set of illumination-invariant features derived from the modified images captured by the camera of the mobile device and a second set of illumination-invariant features derived from images captured by the camera of the mobile robot.
1…comparing features of a room derived from the images captured by the camera of the mobile device with features of the room derived from the images captured by the camera of the mobile robot
US Patent 11669086 does not claim limitations of claim 5.
Maxwell (US 20070242878) teaches of advantage of using illumination invariant versions of images for robot navigation ([0002], disclosing illumination invariant enables robot to operate regardless of presence of shadows)
It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to modify US patent 11669086 to claim wherein the features of the room comprise a first set of illumination-invariant features derived from the modified images captured by the camera of the mobile device and a second set of illumination-invariant features derived from images captured by the camera of the mobile robot as taught by Maxwell to improve navigation of robot.
6. The method of claim 1, further comprising:
receiving data associated with an environment of the mobile robot, the data indicative of a location of the mobile robot determined by the mobile device; and
overlaying the one or more graphic elements on the user interface of the mobile device, wherein the one or more graphic elements are derived from the received data.
6. The method of claim 1, comprising:
at the mobile device, receiving data about an environment from the mobile robot, the data including location information, and
at the mobile device, displaying an image of the environment on a display module, and
overlaying graphic elements on the image in which the graphic elements are derived based on the data received from the mobile robot.
7. The method of claim 6, wherein the one or more graphic elements comprise a representation of at least one of an area processed by the mobile robot within a specified period of time, a keep-out zone, an area traversable by the mobile robot, a restricted area, a signal strength map, or a foot-traffic map.
8. The method of claim 7 in which the area having the specified characteristic represents at least one of (i) an area in the environment that has been processed by the mobile robot within a specified period of time, (ii) an area in the environment that has not been processed by the mobile robot within a specified period of time, (iii) an area in the environment that is scheduled to be processed and has not been processed by the mobile robot, (iv) a keep out zone, (v) an area that is traversable by the mobile robot, (vi) an area that is not traversable by the mobile robot, (vii) a signal strength map, or (viii) a foot traffic statistics map.
8. The method of claim 6, further comprising:
determining a position of an area and a position of an obstacle within the environment, the position of the area and the position of the obstacle being relative to a viewpoint of the camera of the mobile device;
determining which portions of the area are occluded by the obstacle; and
displaying a representation of the environment on the user interface with the portions of the area hidden behind a representation of the obstacle.
10. The method of claim 9, comprising:
determining positions of the area and the obstacle relative to the point-of-view of the camera of the mobile device, determining which portion of the area is occluded by the obstacle, and
displaying the image on the display module with the portion of the area having the specified characteristic being occluded by the obstacle.
9. The method of claim 1, further comprising:
determining first coordinates of a point within a real-world environment of the mobile robot based at least on the first coordinate system; and
converting the first coordinates of the point in the real-world environment to second coordinates of the point in the real-world environment based at least on the determined mapping between the first coordinate system and the second coordinate system.
US Patent 11669086 does not claim limitations recited in claim 9
Forsberg (US 20200081451) teaches of converting coordinates of image to real world coordinates ([0020], disclosing image coordinates of a robot cleaning device are converted into real world height and distance i.e., real world coordinates)
It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to US Parent 11669086 to determining first coordinates of a point within a real-world environment of the mobile robot based at least on the first coordinate system; and
converting the first coordinates of the point in the real-world environment to second coordinates of the point in the real-world environment based at least on the determined mapping between the first coordinate system and the second coordinate system as means to navigate in the work area.
10. The method of claim 1, wherein a floor surface of the room is to be cleaned by the mobile robot during a cleaning operation.
1…wherein a floor surface of the room is to be cleaned by the mobile robot during a cleaning operation
11. The method of claim 1, further comprising:
executing an augmented reality tool to implement an augmented reality session on the mobile device; and
configuring the user interface to enable the user to interact with the mobile device in the augmented reality session
11. The method of claim 1, comprising:
executing an augmented reality tool to implement an augmented reality session on the mobile device, and
configuring the user interface to enable the user to interact with the mobile device in the augmented reality session.
12. The method of claim 11, further comprising using the augmented reality tool to establish the first coordinate system based on the images captured by the camera of the mobile device.
12. The method of claim 11, comprising using the augmented reality tool to establish the first coordinate system based on the images captured by the camera of the mobile device.
Claims 13-20 recite limitations similar in scope to claims 1-12, hence are similarly rejected.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARSLAN AZHAR whose telephone number is (571)270-1703. The examiner can normally be reached Mon-Fri 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARSLAN AZHAR/Examiner, Art Unit 3656
Prosecution Insights