DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1 - 31 are presented for examination.
Claims 1 - 31 are rejected under 35 U.S.C. 112(b).
Claims 1 - 31 are rejected under 35 U.S.C. 101.
Claims 1, 5 -8, 10- 12, 14-16, 18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021)
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), in the view of Sensopia Inc.; "Magicplan - 2D/3D Floor Plans on the App Store";2019; pgs. 1-2, further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), in the view of Sensopia Inc.; "Magicplan - 2D/3D Floor Plans on the App Store";2019; pgs. 1-2, further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages,further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013.
Claims 9 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021) further in the view of Fathi; Habib (US 11106911 B1).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Artiano, JR (US20200257832 A1).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of GANIHAR (US 20210232719 A1).
Claims 22, 24, 26- 29 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Bhogal; Nikhil (US 20120293607 A1)
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages. Artiano, JR (US20200257832 A1).
Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages, further in the view of Fathi; Habib (US 11106911 B1).
This action is Non-Final rejection.
Information Disclosure Statement
The IDS filed on 06/16/2022 and 02/26/2024 are reviewed and considered. See attached file.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, “predicted amount
of time for the producing of the mapping information” must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 - 31 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “further improvement” in claim 1, 5,10 and 24 is a relative term which renders the claim indefinite. The term “further improvement” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. “Further improvement” makes the claim limitations vague and unclear and leave the reader in doubt about the meaning of the word which they referring makes the claim unclean.
There rest of the claims are also dependent on either of those claims so, dependent claims are also rejected on the same rational.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-31 are rejected under 35 U.S.C. 101 because the claimed invention is directed to mental process without any additional element that provide a practical or amount to significant more than the abstract idea.
Step 1: Yes: the claims 1- 9 recites a method and 10-23 recites a system, so claims 1-23 falls into a statutory category of a process and 24-31 recites apparatus.
Step 2A Prong 1: Yes: claims 1-31 recites abstract idea, abstract ideas in each claim are bolded.
Regarding Claim 1:
analyzing, by the , first visual data of the one or more first
panorama images to assess first attributes of the first visual data affecting an
automated floor plan generation process that generates a floor plan for the
building and is based on the first visual data (under its broadest interpretation a human mind can assess attribute from the first visual data, since this limitation doesn’t specify any specific way to assess attribute from visual data. Therefore this limitation recites a mental process because a human mind can assess attribute from a visual data using a pen and paper by observation, evaluation and judgment)
predicting, by the and based at least in part on the
assessed first attributes of the first visual data, one or more characteristics of the
automated floor plan generation process that include a predicted amount of time to complete generation of the floor plan for the building;( under its broadest interpretation a human mind can draw a floor plan based on visual data attribute and can estimate the amount of time it takes to complete the plan by recording the starting and ending time by using a pen and paper. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
providing, by the , feedback to the user that includes the
predicted amount of time and information about the assessed first attributes of the first visual data, to cause improvements in the automated floor plan generation process from acquiring of the additional panorama images by the user ( under its broadest interpretation a human mind give a feedback including the amount of time and assessed information to make improvement on the process. A human mind can make evaluation of the process and time it takes to make a judgment for improvement. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
repeatedly updating, by the , the predicted one or more
characteristics of the automated floor plan generation process based on further
images acquired for the building, including (this claim limitation recites a mental process because a human mind can predict one or more characteristics of the automated floor generation process based on the gathered information by observation , evaluation and judgment with the help of pen and paper in repeated way).
analyzing, by the , second visual data of the one or
more second panorama images to assess second attributes of the second visual
data that further affect the automated floor plan generation process, (under its broadest interpretation a human mind can assess attribute from the second visual data, since this limitation doesn’t specify any specific way to assess attribute from visual data. Therefore this limitation recites a mental process because a human mind can assess attribute from a visual data using a pen and paper by observation, evaluation and judgment)
predicting, by the and based at least in part on the
assessed second attributes, one or more updated characteristics of the automated floor plan generation process that include a revised predicted amount of time to complete the generation of the floor plan for the building (under its broadest interpretation a human mind can estimate the revised amount of time it takes to complete the plan by recording the starting and ending time by using a pen and paper or physical aid. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
providing, by the , further feedback to the user that
includes the revised predicted amount of time and information about the assessed second attributes of the second visual data; to cause further improvements in the automated floor plan generation process ( under its broadest interpretation a human mind give a further feedback including the revised amount of time and assessed information to make further improvement on the process. A human mind can make evaluation of the process and time it takes to make a judgment, feedback for improvement. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
Regarding dependent claims 2-4, it further defines the abstract ideas.
Regarding claim 2
before the repeated updating, generating, by the and for
each of the one or more first rooms, an initial estimate of a room shape for that
first room using the first visual data, (this claim limitation recites a mental process because a human mind can make observation and evaluation on the visual data and make a judgment about the shape of the room by using pen and paper)
for each of the multiple iterations, generating, by the and
for each of the at least one second rooms at which the at least one second
panorama image for that iteration is acquired, an initial estimate of a room shape for that at least one second room using the second visual data,(this claim limitation recites a mental process because a huma mind can repeatedly estimate the shape of rooms by observation, evaluation and judgment with the help of a pen and paper).
wherein generating of the predicted amount of time to complete the
generation of the floor plan for the building is based at least in part on first room
shape uncertainties associated with the generated initial estimates of the room
shapes for each of the first rooms (this claim limitation recites a mental process because a human mind can draw a floor plan based on the shape of the room uncertainty and record the amount of time it takes; to predict the amount of time it takes by observation, evaluation, and judgment with the help of pen and paper).
wherein generating of the revised predicted amount of time for each of
the multiple iterations is based at least in part on second room shape uncertainties associated with the generated initial estimates of the room shapes for each of the at least one second rooms for that iteration( this claim limitation recites a mental process because a human mind can draw a floor plan based on the shape of the room uncertainty and record the amount of time it takes iteratively; to predict the revised amount of time it takes by observation, evaluation, and judgment with the help of pen and paper).
Regarding claim 3
performing, by the , the automated
floor plan generation process using the first visual data and using the second
visual data of the one or more second panorama images acquired for each of the
multiple iterations, including to determine final room shapes for the multiple rooms and to generate the floor plan for the building using the final room shapes;( this claim limitation recites a mental process because a human mind can repeatedly analyze the visual data and determine the final shape of the room to draw the floor plan by observation, evaluation and judgment with the help of pen and paper).
Regarding claim 4
before the acquiring of the one or more first panorama images, determining, by the , computing resources available to the server computing system at a current time for the automated floor plan generation process;(this limitation recites a mental process because a human can make observation on the system to determine the available computing resource at a given time).
Regarding claim 5
analyzing, by the one or more visual data of the one or
more first images to assess one or more attributes of the one or more first images;(this claim limitation recites a mental process because a human mind can analyze visual date of image to assess attributes of images by observation, evaluation and judgment).
predicting, by the one or more and based at least in part
on the assessed one or more attributes of the one or more first images, one or
more characteristics of an automated generation process to produce a floor plan
for the building, wherein the automated generation process is based on analysis of the visual data of the one or more first images and is based on future analysis of additional images to be acquired at additional acquisition locations for the building, and wherein the predicted one or more characteristics include a predicted amount of time to complete the floor plan for the building;
(this claim limitation recites a mental process because a human mind can predict characteristics of the automated process based on assessed attributes of images and the analysis of the visual data by observation, evaluation and judgment).
analyzing, by the one or more visual data of the one or
more second images to assess at least one attribute of the one or more second
images;( under its broadest interpretation a human mind can assess attribute from the second visual data, since this limitation doesn’t specify any specific way to assess attribute from visual data. Therefore this limitation recites a mental process because a human mind can assess attribute from a visual data using a pen and paper by observation, evaluation and judgment)
predicting, by the one or more and based at least in part
on the assessed at least one attribute of the one or more second images, one or
more updated characteristics of the automated generation process that include a
revised predicted amount of time to complete the floor plan for the building; (under its broadest interpretation a human mind can draw a floor plan based on the updated visual data attribute and can estimate the revised amount of time it takes to complete the plan by recording the starting and ending time by using a pen and paper. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
Regarding the dependent claims 6-9, it also further defines the abstract claims
Regarding claim 6
wherein the analyzing of the visual data of the one or more first images is performed by the (this claim limitation recites a mental process because a human mind can analyze the visual data of image by observation, evaluation and judgment).
determining, for each of one or more first rooms that include the first acquisition locations, an initial estimated room shape of that first room, and wherein the providing of the feedback to the one or more users is performed by the
Regarding claim 7, 15 and 28
generating, by the one or more additional and as part of
the automated generation process, the floor plan for the building using the transmitted data, including determining final room shapes for the multiple rooms, and combining the final room shapes for the multiple rooms to complete the floor
plan ( this claim limitation recites a mental process because a human mind can generate a room shape using a gathered data by observation, evaluation and judgment and combine the room shape to create a complete floor plan by using pen and paper).
Regarding claim 9, 17 and 30
predicting, by the one or more and based at least in part
on one or more assessed attributes of the one first image, an adjusted predicted
amount of time to complete the floor plan for the building that is less than the
predicted amount of time based at least in part on the one or more improvements;
(this claim limitation recites a mental process because a human mind can make judgment on the amount of time to complete the plan using assessed attributes of image is less after improvement, so a human can perform this limitation by performing observation, evaluation and judgment to predict the amount of time to complete the floor plan).
Regarding claim 10 and 24
analyzing visual data of the one or more first images to assess one or more attributes of the one or more first images;(this claim limitation recites a mental process because a human mind can analyze visual date of image to assess attributes of images by observation, evaluation and judgment)
generating, based at least in part on the assessed one or more attributes of the one or more first images, one or more predicted characteristics of a generation process for producing mapping information for the building that include a predicted amount of time for the producing of the mapping information, wherein the generation process is based at least in part on analysis of the visual data of the one or more first images and on future analysis of additional images to be acquired at additional acquisition locations for the building;(
(this claim limitation recites a mental process because a human mind can predict characteristics of a generation process including predicting amount of time by making judgment and evaluation or by recording the time it takes by using physical aid using human’s observation and evaluation skill based on analyzed visual data).
analyzing visual data of the one or more second images to assess at least one attribute of the one or more second images; (this claim limitation recites a mental process because a human mind can analyze visual date of image to assess attributes of images by observation, evaluation and judgment)
generating, based at least in part on the assessed at least one attribute of the one or more second images, one or more updated predicted characteristics of the generation process that include a revised predicted amount of time for the producing of the mapping information for the building; (under its broadest interpretation a human mind can generate the updated predicted characteristics of the generation process, including the amount of time it takes for mapping information using the assessed attributes. Therefore this limitation recites a mental process which can be made by a human mind by observation, evaluation and judgment).
using at least the revised predicted amount of time to enable further improvement in the generation process (this claim limitation recites a mental process because a human mind can use the revised amount of time in the process and make evaluation and judgment on the process for improvement).
Regarding dependent claims 11-31 further defines abstract ideas.
Regarding claim 12
providing the feedback to one or more users associated with acquiring of the one or more first images and acquiring of the additional images and includes providing information about the assessed one or more attributes of the one or more first images, wherein the assessed one or more attributes of the one or more first images include at least one assessment of quality of the visual data of the one or more first images and wherein the using of the at least revised predicted amount of time includes providing revised feedback to the one or more users that includes the revised predicted amount of time, to enable further improvement in producing the floor plan ( under its broadest reasonable interpretation, this claim limitation recites a mental process because a human mind can provide feedback and assess attributes by making observation, evaluation and judgment on the first image and additional images).
Regarding claim 14
wherein the analyzing of the visual data of the one or more first images is performed at least in part by the (this claim limitation recites a mental process because a human mind can analyze the visual data of image by observation, evaluation and judgment).
and includes determining, for each of the one or more first rooms, an initial estimated room shape of that first room, and wherein the providing of the feedback is performed by the ( this claim limitation recites a mental process because a human mind can estimate the shape of the room bay observation and evaluation and a human mind can also give a feedback by observation, evaluation and judgment).
Regarding claim 18
determining, for each of the first and additional images, acquisition pose
information based at least in part on the motion data acquired by the one or more
during acquiring of that image and on visual data of that image, and wherein the determining of the initial estimated room shape for each of
the one or more first images is based in part on the respective determined
acquisition pose information for each of the one or more first images (under its broadest reasonable interpretation a human mind can determine the pose information based on the motion data using a physical aid by making observation, evaluation and judgment and based this information a human mind can also estimate the room shape by evaluating and observing the pose information and sensor is used to gather information see step 1A prong 2 analysis).
Regarding claim 20
wherein the analyzing of the visual data of the one or more first images includes identifying one or more problems to correct that include at least one of a lack of inter-image line-of-sight between visual data of at least one of the first images and at least one other acquired image, or a lack of overlap between visual data of at least one of the first images and at least one other acquired image ….(this claim limitation recites a mental process because a human mind can identify a problem on image by observation and judgment, for example a human can observe the image to see if there is a lack of coverage of all of the multiple rooms).
Regarding claim 21
wherein generating of predicted and revised predicted amounts of time includes …. ( this claim further defines abstract idea of generating a predicted amount of time see claim 10 above)
Regarding claim 22
determining computing resources available for implementing the generation
process;(this claim recites a mental process because a human mind can observe and evaluate available computational resource before performing generation)
determining an initial prediction of an amount of time for the producing of
the mapping information for the building that is based at least in part on the
determined computing resources and on information about the one or more users
and on publicly available information about the building;( this claim limitation recites a mental process because a human mind can determine the amount of time to produce the mapping information by using publicly available information and computing resource by evaluation and judgement).
Regarding claim 23
determining, based at least in part on the one or more first images and
before acquiring of the one or more second images, a current status of acquiring
images for the generation process that includes one or more characteristics of at
least one of an amount completed of the acquiring of the images for the generation process or an amount remaining of the acquiring of the images for the generation process and wherein the providing of the feedback includes providing information to one or more users that includes the predicted amount of time and the determined current status including the one or more characteristics,( this limitation recites a mental process because a human mind can determine the amount of time remains based on images to determine the current status of acquiring image by making observation on the process or by making evaluation and judgment on the process and a human mind can make judgments based on the observation to give a feedback).
determining, based at least in part on the one or more second images, a
revised current status of acquiring images for the generation process that includes revisions to the one or more characteristics, and wherein the providing of the further feedback includes providing information to the one or more users that includes the revised predicted amount of time and the determined revised current status ( this limitation recites a mental process because a human mine can determine the revised status by observing the process of acquiring image based on the second image and a human mind can also make evaluation and judgment to provide the provide the updated feedback).
Regarding claim 27
the analyzing of the visual data of the one or more first images is performed by the and includes determining, for each of the one or more first rooms, an initial estimated room shape of that first room, and wherein the providing of the feedback to the one or more users is performed by the because a human mind can analyze the visual data by making observation, evaluation and judgment, and a person can also determine the initial shape and provide a feedback based on evaluation using a pen and paper).
Regarding claim 30
predicting, by the one or more and based at least in part
on one or more assessed attributes of the one first image, an adjusted predicted
amount of resources involved with the producing of the floor plan for the building
that is less than the predicted amount of resources based at least in part on the
one or more improvements(this claim further defines a mental process because a human mind can make observation on the system to evaluate the amount of computational resource used in the planning process by observation , evaluation and judgment).
Regarding claim 31
predicted one or more characteristics of the automated generation process for producing the floor plan for the building include a predicted amount of time… this claim limitation further defines the abstract idea of predicting the amount of time and giving a feedback to the user. A human mind can make prediction by making observation and evaluation on the image acquisition process and make a judgment based it to provide feedback with the help of pen and paper.
The dependent claims 12 and 19 also further defines the abstract ideas of mapping information by narrowing the scope claim 10 as it is explained above on claim 10.
Step 2A prong 2: No
The claims do not recite additional elements that integrate the exception into a practical application of the exception because the claim do not have additional elements or a combination of additional elements that apply, rely on, or use the judicial exception in a manner that impose a meaningful limit on the judicial exception.
Claims recites gathering data which is insignificant extra solution activity. Adding
insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea such as a step of obtaining information about credit card transactions so that the information can be analyzed by an abstract mental process, as discussed in CyberSource V. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011) (see MPEP § 2106.05(g), and claims also recites data manipulation by “displaying” outputs - Selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016); MPEP 2106.05(g).
The claim limitations which recites data gatherings, and manipulation are listed below
Claim 1
acquiring, by (insignificant extra-solution activity – data gathering such as 'obtaining information'. See MPEP 2106.05(g).).
and on further visual data to be acquired in additional panorama images at later times at additional acquisition locations in additional rooms for the building(insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g). )
acquiring, by the
images of the additional panorama images at one or more additional second acquisition locations for the building, wherein at least one of the second panorama images are acquired in at least one second room of the multiple rooms and have visual coverage of at least some walls of the at least one second room;( insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
the second attributes being based in part on an overlap of the second visual data with other visual data of one or more prior panorama images and being based in part on quality of the second visual data; ;(insignificant extra-solution activity – data manipulation)
Claim 2
wherein the providing of the feedback and of the further feedback includes the feedback and the further feedback in (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
displaying, together with the further feedback to the user, a further visual representation for each of the at least one second rooms of the generated initial estimate of the room shape for that at least one second room ;(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
displaying, together with the feedback to the user, a visual representation for each of the one or more first rooms of the generated initial estimate of the room shape for that first room;(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claim 3
transmitting, by the
systems over one or more
images( insignificant extra solution activity- WURC, MPEP 2106.05(d)(II) by at least "I. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321)
for each of the multiple iterations, the one or more second panorama
images acquired for that iteration; (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g). )
transmitting, by the one or more
plan for the building(insignificant extra solution activity- WURC, MPEP 2106.05(d)(II) by at least "I. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321)
Claim 4presenting, by the
amount of time to complete the generation of the floor plan for the building that is
determined based at least in part on the determined computing resources and on
information about past performance of the user in acquiring images and on
publicly available information about the building ;(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claim 5
obtaining, by one or more
multiple rooms, one or more first images acquired by one or more users at one or
more first acquisition locations in a subset of the multiple rooms, wherein each of the first images has visual coverage of at least some walls of one of the multiple rooms in which the respective first acquisition location for that first image is located;(insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
presenting, by the one or more
more users that includes at least the predicted amount of time, to enable
improvement in the additional images to be acquired;(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
obtaining, by the one or more
images of the additional images that are acquired by the one or more users at one
or more additional second acquisition locations for the building; (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g)).
presenting, by the one or more computing devices, revised feedback to the
one or more users that includes at least the revised predicted amount of time, to
enable further improvement in the automated generation process ;(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claims 6 and 14
displaying information on the image acquisition computing device that includes the predicted amount of time and the initial estimated room shape for each of the one or more first rooms and one or more indications of aspects of the one or more first images to change during acquisition of the additional images to produce the improvement in the additional images (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claims 7, 15 and 28
transmitting, by the image acquisition
more networks to the one or more additional
information from the one or more first images and the additional images;( insignificant extra solution activity- WURC, MPEP 2106.05(d)(II) by at least "i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321)
Claims 8 and 28
wherein the providing of the feedback to the one or more users by the image acquisition computing device further includes displaying instructions to correct one of the first images based at least in part on at least one assessed attribute of the one first image that reflects quality of the visual data of the one first image (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claims 9 and 17
acquiring, by the image acquisition
the feedback, a new image to replace the one first image that includes one or
more improvements to correct the one first image;(insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).
displaying, by the image acquisition computing device, the adjusted
predicted amount of time (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claim 10 and 24
obtaining, for a building with multiple rooms, one or more first images
acquired at one or more first acquisition locations in one or more first rooms of the multiple rooms, wherein each of the first images has visual coverage of at least some walls of one of the first rooms that includes the respective first acquisition location for that first image; (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g))
obtaining one or more second images of the additional images that are acquired at one or more additional second acquisition locations for the building; (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g)).
Claim 13
displaying the feedback and the revised feedback in a (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
includes displaying the completed floor plan in the graphical user interface
(insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claim 16
wherein the generating of the floor plan for the building further includes obtaining input from one or more users related to at least one of the final room shapes for the multiple rooms or the combining of the final room shapes, and using the obtained input as part of the generating of the floor plan.(insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g)).
Claim 18
wherein the first and additional images are each a panorama image having 360 degrees of horizontal visual coverage around a vertical axis, wherein acquiring of the first and additional images is performed without using any depth information from any depth-sensing sensors for distances to surrounding surfaces and includes using one or more .(insignificant extra-solution activity – data gathering, it further specifies gathered information( first and additional images)).
Claim 22
wherein acquiring of the first and additional images for the building is performed by one or more users, wherein the using of the at least revised predicted amount of time includes providing further feedback that includes at least the revised predicted amount of time, and wherein the automated operations further include, before acquiring any images of the building for use in the producing of the mapping information for the building (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
presenting to the one or more users the determined initial prediction of the
amount of time, wherein the providing of the feedback and of the revised feedback is performed to provide repeated updates to the one or more users of a predicted amount of time based on cumulative acquisition of images for the building. (insignificant extra-solution activity – data gathering, such as “outputting data’'. See MPEP 2106.05(g).)
Claim 25
wherein the one or more (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
Claim 26
displaying information on the image acquisition (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Claim 27
wherein the one or more images and the additional images (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
Claim 30
acquiring, by the image acquisition
the feedback, a new image to replace the one first image that includes one or
more improvements to correct the one first image; (insignificant extra-solution activity – data gathering, such as 'obtaining information'. See MPEP 2106.05(g).)
displaying, by the image acquisition computing device, the adjusted
predicted amount of resources involved with the producing of the floor plan for the building (insignificant extra-solution activity – data gathering, such as 'outputting data'. See MPEP 2106.05(g).)
Step 2B: No
The claims do not cite additional elements which are significantly more than the abstract idea. As outlined above the claims merely use a computer components as a tool to perform abstract ideas and it also use additional elements like camera and sensor to obtain or gather information. Merly using of a computer and applying abstract ideas into a system without making improvement to the functionality of a computer is not a significantly more.
The dependent claims include the same abstract ideas as recited in the
independent claim and merely incorporate additional details that narrow the scope of abstract ideas and fails to add significantly more than the claims
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more. The claims recites a computing device, including GUI, processors, and memories, used to obtain , transmit and store are identified above as insignificant extra solution activity are WURC as evidenced by the court cases cited in MPEP
2106.05(d)(II) by at least "i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, buySAFE, Inc. V. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)" and "iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. V. SAP Am., Inc., 793 F.3d 1306, OIP Techs., 788 F.3d at 1363
-"presenting" identified above as insignificant extra solution activity above is also WURC as evidenced by the court cases cited in MPEP 2106.05(d)(II) by at least "iv. Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-9" for Berkheimer support of being well-understood, routine, and conventional computer outputting of data.
-"a non-transitory computer-readable medium having stored contents that cause one or more computing systems to perform automated operations" which are all a high- level recitation of a generic computer components and represent mere instructions to apply the judicial exception on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Though the claim recites additional elements as it is outlined above, they are not significantly more than abstract idea since the additional elements are merely used as a tool and it does not make improvement to the functioning of the additional elements, see MPEP 2106.05(a), Improvements to the Functioning of a Computer or To Any Other Technology or Technical Field [R-07.2022]
Therefore, it is concluded that the claims 1-31 are not found eligible under 35 U.S.C 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5 -8, 10- 12, 14-16, 18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021)
As of claim 1, Li teaches A computer-implemented method (Para 58, Accordingly, embodiments of the present disclosure may be practiced with other computer system configurations).
acquiring, by a computing device having one or more cameras and under control of a user, one or more first panorama images at one or more first acquisition locations in one or more first rooms of a building that has multiple rooms, wherein each of the first panorama images is in a spherical format and includes 360 degrees of horizontal visual coverage around a vertical axis( para 09 In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 2D (two- dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 3600 spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis) ) and has visual coverage of at least some walls of one of the first rooms that contains the respective first acquisition location for that first panorama image(para 25, In particular, Figure 2A illustrates an example image 250a, such as a perspective image taken in a northeasterly direction from viewing location 210B in the living room of house 198 of Figure 1B (or a northeasterly facing subset view of a 360-degree panorama image taken from that viewing location and formatted in a rectilinear manner)…. In the illustrated example, the displayed image includes built-in elements (e.g., light fixture 130a), furniture (e.g., chair 192- 1 ), two windows 196-1, and a picture 194-1 hanging on the north wall of the living room. No inter-room passages into or out)
analyzing, by the computing device, first visual data of the one or more first
panorama images to assess first attributes of the first visual data affecting an
automated floor plan generation process that generates a floor plan for the
building and is based on the first visual data(para 11, Additional details are included below regarding automated operations of device(s) implementing an Image Capture and
Analysis (ICA) system involved in acquiring images and optionally acquisition metadata, as well as in optionally performing preprocessing of the images before later use (e.g., to render 360° spherical panorama images in an equirectangular format
[0012] In some embodiments, one or more types of additional processing may be further performed, such as to determine additional mapping-related information for a generated floor map or to otherwise associate additional information with a generated floor map…. As another example, in at least some embodiments, additional processing of images is performed to determine estimated distance information of one or more types, such as to measure sizes in images of objects of known size, and use such information to estimate room width, length and/or height dimensions, [79], a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building) and on further visual data to be acquired in additional panorama images at later times at additional acquisition locations in additional rooms for the building(para 12, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations, other audio information, such as recordings of ambient noise; overall dimension information, etc.
repeatedly updating, by the computing device, the predicted one or more
characteristics of the automated floor plan generation process based on further
images acquired for the building, including, for each of multiple iterations: (fig 4 , label 425, Optionally analyze viewing location information to identify possible additional coverage and/or other information to gather, provide corresponding user suggestions, and gather additional information as indicated, label 477 stores images and any associated generated /obtained information), as shown on figure 4, the process of image generation is also repetitive and 422 obtain acceleration (characteristics of generation process) and label 424 performs updating.
acquiring, by the computing device, one or more second panorama images of the additional panorama images at one or more additional second acquisition locations for the building, (para 61, After block 415 is completed, the routine continues to block 420 to determine if there are more viewing locations at which to acquire images, such as based on corresponding information provided by the user of the mobile device. If so, and when the user is ready to continue the process, the routine continues to block 422 to optionally initiate the capture of linking information (including acceleration data) during movement of the mobile device along a travel path away from the current viewing location and towards a next viewing location within the building interior)wherein at least one of the second panorama images are acquired in at least one second room of the multiple rooms and have visual coverage of at least some walls of the at least one second room;(para 75, In block 550, the routine receives a first selection (e.g., from an operator user of the MIGM system) of a first 360°
spherical panorama image with an equirectangular projection taken in a first room of the building and a second selection (e.g., from the operator user of the MIGM system) of a second 360° spherical panorama image with an equirectangular projection taken in a second room of the building, and/or first and second selections of the first room of the building and the second room of the building, Fig.2B and 2C, shows the visual coverage of at least some wall)
analyzing, by the computing device, second visual data of the one or more second panorama images to assess second attributes of the second visual data that further affect the automated floor plan generation process,( fig 5A label 530 and 535, Receive user indications of doors, inter-room openings and/or windows on image in first GUI portion and/or on room shape in second GUI portion and dynamically
adjust both room outline and room shape to display visual indicators and reflect user manipulations, optionally receive user annotations or other non-image information to link to room or to locations within room and add corresponding visual indicators, optionally apply automated optimization(s) to room shape based on geometrical constraints and/or other information, optionally determine room dimensions from one or more known or estimated lengths, and store final room shape along with other determined/received information).
Li does not explicitly teach predicting, by the computing device and based at least in part on the assessed first attributes of the first visual data, one or more characteristics of the automated floor plan generation process that include a predicted amount of time to complete generation of the floor plan for the building, providing, by the computing device, feedback to the user that includes the predicted amount of time and information about the assessed first attributes of the first visual data, to cause improvements in the automated floor plan generation process from acquiring of the additional panorama images by the user, the second attributes being based in part on an overlap of the second visual data with other visual data of one or more prior panorama images and being based in part on quality of the second visual data; predicting, by the computing device and based at least in part on the assessed second attributes, one or more updated characteristics of the automated floor plan generation process that include a revised predicted amount of time to complete the generation of the floor plan for the building and providing, by the computing device, further feedback to the user that includes the revised predicted amount of time and information about the assessed second attributes of the second visual data, to cause further improvements in the automated floor plan generation process.
While Pham teaches predicting, by the computing device and based at least in part on the assessed first attributes of the first visual data, one or more characteristics of the automated floor plan generation process that include a predicted amount of time to complete generation of the floor plan for the building; (Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4, “processing time”
As shown in Figure 12, we evaluate the performance of the proposed method and five cutting-edge stitching software, in terms of the total processing time for the entire stitching process with the number of images of the Technology Park scene varying from 10 to 200. The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, especially for Pix4d [35], Hugin [42], and Photoshop [43]. The pre-processing time of the proposed method is also added for a fair comparison. The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively).
providing, by the computing device, feedback to the user that includes the
predicted amount of time and information about the assessed first attributes of the first visual data, to cause improvements in the automated floor plan generation process from acquiring of the additional panorama images by the user; (Abstract The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods … Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section III, Additionally, it is recommended that aerial images be captured with at least 80%-85% overlap to obtain the best output quality for forest
or densely vegetated areas [35], [36]. If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate).
the second attributes being based in part on an overlap of the second visual data with other visual data of one or more prior panorama images and being based in part on quality of the second visual data; ( section III, Additionally, it is recommended that aerial images be captured with at least 80%-85% overlap to obtain the best output quality for forest or densely vegetated areas [35], [36]… Specially, if two images overlap, we estimate their overlapping ratio and specify one image's position and overlapping region relative to those of the other. To this end, the geotagging metadata information of the UAV images, including GPS coordinates, UAV altitude, camera gimbal yaw angle, camera angle of view, and image resolution is exploited to estimate the footprint of the images on the ground).
predicting, by the computing device and based at least in part on the assessed second attributes, one or more updated characteristics of the automated floor plan generation process that include a revised predicted amount of time to complete the generation of the floor plan for the building; and (
PNG
media_image1.png
567
711
media_image1.png
Greyscale
As shown above on the graph , for different image number it have updated amount of time it takes for stitching using different model and the proposed model use image attributes like number of images and resolution. (Section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively), as shown in the graph the second number of images(20) is considered as the second images with attribute if the number of images.
providing, by the computing device, further feedback to the user that includes the revised predicted amount of time and information about the assessed second attributes of the second visual data, to cause further improvements in the automated floor plan generation process.( Abstract, The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods
Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4 The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, Figure 12 show feedback including time , based on resolution and number of images).
Li and Pham are considered to be analogous to the claimed invention since they focus on automatic operation of involved image analysis and combining images to create a complete scene. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to combine Pham teaching of estimating amount of time based on different attributes in to Li ‘s model in order to further provide benefits in allowing improved automated navigation of a building by mobile devices (e.g., semi-autonomous or fully autonomous vehicles), including to significantly reduce their computing power used and time used to attempt to otherwise learn a building's layout (Li, para 13) and improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods( Pham, abstract).
As of claim 5, Li teaches A computer-implemented method (Para 58, Accordingly, embodiments of the present disclosure may be practiced with other computer system configurations).
obtaining, by one or more computing devices and for a building with multiple rooms, one or more first images acquired by one or more users at one or more first acquisition locations in a subset of the multiple rooms, (para 09, In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 2D (two- dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 3600 spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis)
wherein each of the first images has visual coverage of at least some walls of one of the multiple rooms in which the respective first acquisition location for that first image is located;(para 25, In particular, Figure 2A illustrates an example image 250a, such as a perspective image taken in a northeasterly direction from viewing location 210B in the living room of house 198 of Figure 1B (or a northeasterly facing
subset view of a 360-degree panorama image taken from that viewing location and formatted in a rectilinear manner)…. In the illustrated example, the displayed image includes built-in elements (e.g., light fixture 130a), furniture (e.g., chair 192- 1 ), two windows 196-1, and a picture 194-1 hanging on the north wall of the living room. No inter-room passages into or out of the living room (e.g., doors or other wall openings) are visible in this image).
analyzing, by the one or more computing devices, visual data of the one or
more first images to assess one or more attributes of the one or more first images; (para 11, Additional details are included below regarding automated operations of device(s) implementing an Image Capture and Analysis (ICA) system involved in acquiring images and optionally acquisition metadata, as well as in optionally performing preprocessing of the images before later use (e.g., to render 360° spherical panorama images in an equirectangular format).
[0012] In some embodiments, one or more types of additional processing may
be further performed, such as to determine additional mapping-related information for a generated floor map or to otherwise associate additional information with a generated floor map…. As another example, in at least some embodiments, additional processing of images is performed to determine estimated distance information of one or more types, such as to measure sizes in images of objects of known size, and use such
information to estimate room width, length and/or height dimensions.
wherein the automated generation process is based on analysis of the visual data of the one or more first images and is based on future analysis of additional images to be acquired at additional acquisition locations for the building,(para 12, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations, other audio information, …)
obtaining, by the one or more computing devices, one or more second images of the additional images that are acquired by the one or more users at one or more additional second acquisition locations for the building;(para 61, After block 415 is completed, the routine continues to block 420 to determine if there are more viewing locations at which to acquire images, such as based on corresponding information provided by the user of the mobile device. If so, and when the user is ready to continue the process, the routine continues to block 422 to optionally initiate the capture of linking information (including acceleration data) during movement of the mobile device along a travel path away from the current viewing location and towards a next viewing location within the building interior)
analyzing, by the one or more computing devices, visual data of the one or
more second images to assess at least one attribute of the one or more second
images; (para 75 In some embodiments, the routine may further perform automated operations to identify candidate locations of each inter-room passage between the first and second rooms in both of the first and second images (such as likely locations from image analysis and using machine learning techniques), and then uses that information to guide the layout determination of the room shapes for the first and second rooms).
Li does not explicitly teach predicting, by the one or more computing devices and based at least in part on the assessed one or more attributes of the one or more first images, one or more characteristics of an automated generation process to produce a floor plan for the building, wherein the predicted one or more characteristics include a predicted amount of time to complete the floor plan for the building; presenting, by the one or more computing devices, feedback to the one or more users that includes at least the predicted amount of time, to enable improvement in the additional images to be acquired, predicting, by the one or more computing devices and based at least in part on the assessed at least one attribute of the one or more second images, one or more updated characteristics of the automated generation process that include a revised predicted amount of time to complete the floor plan for the building; and presenting, by the one or more computing devices, revised feedback to the one or more users that includes at least the revised predicted amount of time, to enable further improvement in the automated generation process.
While Pham teaches predicting, by the one or more computing devices and based at least in part on the assessed one or more attributes of the one or more first images, one or more characteristics of an automated generation process to produce a floor plan for the building, (Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4 “processing time”, As shown in Figure 12, we evaluate the performance of the proposed method and five cutting-edge stitching software, in terms of the total processing time for the entire stitching process with the number of images of the Technology Park scene varying from 10 to 200. The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, especially for Pix4d [35], Hugin [42], and Photoshop [43]. The pre-processing time of the proposed method is also added for a fair comparison. The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively)
wherein the predicted one or more characteristics include a predicted amount
of time to complete the floor plan for the building; (section 4 The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively)
presenting, by the one or more computing devices, feedback to the one or
more users that includes at least the predicted amount of time, to enable
improvement in the additional images to be acquired(Abstract The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods … Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section III, Additionally, it is recommended that aerial images be captured with at least 80%-85% overlap to obtain the best output quality for forest or densely vegetated areas [35], [36]. If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate).
predicting, by the one or more computing devices and based at least in part
on the assessed at least one attribute of the one or more second images, one or
more updated characteristics of the automated generation process that include a
revised predicted amount of time to complete the floor plan for the building; and (
PNG
media_image1.png
567
711
media_image1.png
Greyscale
As shown above on the graph , for different image number it have updated amount of time it takes for stitching using different model and the proposed model use image attributes like number of images and resolution. (Section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively) as shown in the graph the second number of images(20) is considered as the second images with attribute if the number of images.
presenting, by the one or more computing devices, revised feedback to the
one or more users that includes at least the revised predicted amount of time, to
enable further improvement in the automated generation process.( Abstract, The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4 The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, Figure 12 show feedback including time , based on resolution and number of images).
Li and Pham are considered to be analogous to the claimed invention since they focus on automatic operation of involved image analysis and combining images to create a complete scene. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to combine Pham teaching of estimating amount of time based on different attributes in to Li ‘s model in order to further provide benefits in allowing improved automated navigation of a building by mobile devices (e.g., semi-autonomous or fully autonomous vehicles), including to significantly reduce their computing power used and time used to attempt to otherwise learn a building's layout (Li, para 13) and improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods( Pham, abstract).
As of claim 6, the combined model teaches all the limitations of claim 5, and Li also teaches wherein the one or more computing devices include an image acquisition computing device with one or more cameras that is used to acquire the one or more first images and the additional images, (para 09, In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 20 (two dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis), …[20] While such a mobile image acquisition device may include various hardware components, such as a camera, one or more sensors (e.g., a gyroscope, an accelerometer, a compass, etc.)
wherein the analyzing of the visual data of the one or more first images is performed by the image acquisition computing device and includes (para 09…floor map that is generated from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis).
determining, for each of one or more first rooms that include the first acquisition locations, an initial estimated room shape of that first room, and wherein the providing of the feedback to the one or more users is performed by the image acquisition computing device(para 11, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations,
[30] After the corner and border visual representations are displayed in the GUI, the user is able to manipulate the visual representations to match corresponding features of the room that are visible in the underlying panorama image, as illustrated in further detail with respect to Figure 2F.(feedback)
[46] As in Figures 2E-2H, a second GUI pane 255s is shown with an initial room shape 260s corresponding to the visual border and corner GUI controls 280, 282 and 285 in the first GUI pane, Label 581 Receive selection of room with defined room shape and image from within room, optionally automatically determine likely image location in room from image analysis and/or metadata, and display GUI with image in sixth GUI portion and room shape in seventh GUI portion)
includes displaying information on the image acquisition computing device (Fig. 1A, display system 142, para 82, including to display images (e.g., 360° spherical panorama images) and/or other information associated with particular locations in the mapping information).and the initial estimated room shape for each of the one or more first rooms (para 29, The second GUI pane 255e displays an initial room shape...)and one or more indications of aspects of the one or more first images to change during acquisition of the additional images to produce the improvement in the additional images (para 62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality).
Pham also teaches displaying the predicted amount of time ( Figure 12) see claim 5 above so it would be obvious for a person of ordinary skill in the art to combine Li and Pham teaching to display the information including amount of time, room shape and aspect to change to produce the improvement in additional images.
Claim 14 is also in the same scope to claim 6, so claim 14, is also rejected under the same rational as of claim 6.
As of claim 7, the combined model teaches all the limitations of claim 6 , and Li also teaches wherein the one or more computing devices further include one or more additional computing devices, and wherein the method further comprises (para
15 Figure 1A is an example block diagram of various computing devices and
systems that may participate in the described techniques in some embodiments. In particular, one or more 360° spherical panorama images 165 in equirectangular format have been generated by an Interior Capture and Analysis ("ICA") system (e.g., a system 160 that is executing on one or more server computing systems 180, and/or a system provided by application 155 executing on one or more mobile image acquisition devices 185),
transmitting, by the image acquisition computing device, data over one or
more networks to the one or more additional computing devices that includes
information from the one or more first images and the additional images;(para
53, The server computing system(s) 300 and executing ICA system 340, and server computing system(s) 380 and executing MIGM system 389, may communicate with each other and with other computing systems and devices in this illustrated embodiment via one or more networks 399 (e.g., the Internet, one or more cellular telephone networks, etc.), such as to interact with user client computing devices 390 (e.g., used to view floor maps, and optionally associated images and/or other related information) and on fig 3 from label 300 and 360 to 390 by using a network 399
generating, by the one or more additional computing devices and as part of
the automated generation process, the floor plan for the building using the
transmitted data, including determining final room shapes for the multiple rooms,
fig 5A label 530 optionally apply automated optimization(s) to room shape based on geometrical constraints and/or other information, optionally determine room dimensions from one or more known or estimated lengths, and store final room shape along with other determined/received information
[45], After all of the room shape layout information has been specified and any such wall width information has been determined, the final results may be used to generate a floor map of the house, [68] Once the user is done, the final room shape in the second GUI portion provides a user-defined estimate of the room shape.
[79] a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building, [claim 14] determining, by the one or more computing devices and for each of the one or more rooms, a final user-defined room shape for the room by combining information received from the user about borders of the room from each of the multiple images taken within the room.
and combining the final room shapes for the multiple rooms to complete the floor
plan; and (para 57, the routine continues to block 545 to determine if the instructions received in block 505 (or subsequently) indicate to combine room shapes of multiple indicated rooms in a building to determine their relative layout).
providing, by the one or more additional computing devices, the floor plan
for the building for further use (para 09, The generated floor map and/or other
generated mapping-related information may be further used in one or more
manners in various embodiments, such as for controlling navigation of mobile
devices (e.g., autonomous vehicles), for display on one or more client devices
in corresponding GUls (graphical user interfaces), etc.).
Claim 15 is also in the same scope to claim 7, so claim 15, is also rejected under the same rational as of claim 7.
As of claim 8, the modified model teaches all the limitations of claim 6, and li also teaches wherein the providing of the feedback to the one or more users by the image acquisition computing device further includes displaying instructions to correct one of the first images based at least in part on at least one assessed attribute of the one first image that reflects quality of the visual data of the one first image (para 62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality, or do not appear to provide complete coverage of the building. After block 425, the routine continues to block 435 to optionally preprocess the acquired 360° spherical panorama images before their subsequent use for generating related mapping information)
As of claim 10, Li teaches one or more hardware processors of one or more computing devices; and one or more memories with stored instructions that, when executed by at least one of the one or more hardware processors, cause at least one of the one or more computing devices to perform automated operations including at least (para 54, In the illustrated embodiment, an embodiment of the ICA system 340 executes in memory 330 of the server computing system(s) 300 in order to perform at least some of the described techniques, such as by using the processor(s) 305 to execute software instructions of the system 340 in a manner that configures the processor(s) 305 and computing system 300 to perform automated operations that implement those described techniques)
obtaining, for a building with multiple rooms, one or more first images acquired at one or more first acquisition locations in one or more first rooms of the multiple rooms, (para 09, In at least some embodiments, the defined area includes an interior
of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 2D (two- dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 3600 spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis)
wherein each of the first images has visual coverage of at least some walls of one of the first rooms that includes the respective first acquisition location for that first image;(para 25, In particular, Figure 2A illustrates an example image 250a, such as a perspective image taken in a northeasterly direction from viewing location 210B in the living room of house 198 of Figure 1B (or a northeasterly facing subset view of a 360-degree panorama image taken from that viewing location and formatted in a rectilinear manner)…. In the illustrated example, the displayed image includes built-in elements (e.g., light fixture 130a), furniture (e.g., chair 192- 1 ), two windows 196-1, and a picture 194-1 hanging on the north wall of the living room. No inter-room passages into or out of the living room (e.g., doors or other wall openings) are visible in this image
analyzing visual data of the one or more first images to assess one or more attributes of the one or more first images; (para 75, In some embodiments, the routine may further perform automated operations to identify candidate locations of each inter-room passage between the first and second rooms in both of the first and second images (such as likely locations from image analysis and using machine learning techniques), and then uses that information to guide the layout determination of the room shapes for the first and second rooms).
wherein the generation process is based at least in part on analysis of the visual
data of the one or more first images and on future analysis of additional images to be acquired at additional acquisition locations for the building; (para
12, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations, other audio information...).
obtaining one or more second images of the additional images that are acquired at one or more additional second acquisition locations for the building; (para
61, After block 415 is completed, the routine continues to block 420 to determine
if there are more viewing locations at which to acquire images, such as based
on corresponding information provided by the user of the mobile device. If so, and when the user is ready to continue the process, the routine continues to block 422 to optionally initiate the capture of linking information (including acceleration data) during movement of the mobile device along a travel path away from the current viewing location and towards a next viewing location within the building interior).
analyzing visual data of the one or more second images to assess at least one attribute of the one or more second images; (para 75, In some embodiments, the routine may further perform automated operations to identify candidate locations of each inter-room passage between the first and second rooms in both of the first and second images (such as likely locations from image analysis and using machine learning techniques), and then uses that information to guide the layout determination of the room shapes for the first and second rooms)
Li does not explicitly teaches generating, based at least in part on the assessed one or more attributes of the one or more first images, one or more predicted characteristics of a generation process for producing mapping information for the building that include a predicted amount of time for the producing of the mapping information, providing feedback that includes at least the predicted amount of time, to enable improvement in the generation process resulting at least in part from the additional images to be acquired; generating, based at least in part on the assessed at least one attribute of the one or more second images, one or more updated predicted characteristics of the generation process that include a revised predicted amount of time for the producing of the mapping information for the building; and using at least the revised predicted amount of time to enable further improvement in the generation process.
While Pham teaches generating, based at least in part on the assessed one or more attributes of the one or more first images, one or more predicted characteristics of a generation process for producing mapping information for the building that include a predicted amount of time for the producing of the mapping information((Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4 “processing time”, As shown in Figure 12, we evaluate the performance of the proposed method and five cutting-edge stitching software, in terms of the total processing time for the entire stitching process with the number of images of the Technology Park scene varying from 10 to 200. The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, especially for Pix4d [35], Hugin [42], and Photoshop [43]. The pre-processing time of the proposed method is also added for a fair comparison. The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively)
providing feedback that includes at least the predicted amount of time, to enable improvement in the generation process resulting at least in part from the additional images to be acquired;(Abstract The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods, Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM,
Figure 12 show feedback including time , for resolution of and number of images).
generating, based at least in part on the assessed at least one attribute of the one or more second images, one or more updated predicted characteristics of the generation process that include a revised predicted amount of time for the producing of the mapping information for the building;(
PNG
media_image1.png
567
711
media_image1.png
Greyscale
As shown above on the graph , for different image number it have updated amount of time it takes for stitching using different model and the proposed model use image attributes like number of images and resolution. (Section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively), as shown in the graph the second number of images(20) is considered as the second images with attribute if the number of images.
using at least the revised predicted amount of time to enable further improvement in the generation process. (Abstract, The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions… section III, If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate[20], [25]. To address this problem, we propose a fast adaptive selection algorithm to eliminate unnecessary input images that are densely overlapped with other images).
Li and Pham are considered to be analogous to the claimed invention since they focus on automatic operation of involved image analysis and combining images to create a complete scene. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to combine Pham teaching of estimating amount of time based on different attributes in to Li ‘s model in order to further provide benefits in allowing improved automated navigation of a building by mobile devices (e.g., semi-autonomous or fully autonomous vehicles), including to significantly reduce their computing power used and time used to attempt to otherwise learn a building's layout (Li, para 13) and improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods( Pham, abstract).
As of claim 11, the combined model teaches all the limitations of claim10, and Li also teaches wherein the mapping information for the building includes a floor plan for the building, (para 65, In the example of Figures 5A-5C, the generated mapping information includes a floor map of a building (e.g., a house), but in other embodiments, other types of mapping information…)
wherein the automated operations further include producing and providing the floor plan for the building ( Figures 2A-2W, illustrate examples of automated operations for participating in analysis of images and generation of a floor map for a building, and [79] it will be appreciated that if sufficiently detailed dimension information is obtained (e.g., as discussed with respect to block 530), a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building).
As of claim 12, the combined model teaches all the limitations of claim 11, and Li also teaches wherein the providing of the feedback includes providing the feedback to one or more users associated with acquiring of the one or more first images and acquiring of the additional images and includes providing information about the assessed one or more attributes of the one or more first images, wherein the assessed one or more attributes of the one or more first images include at least one assessment of quality of the visual data of the one or more first images, (para 62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality, or do not appear to provide complete coverage of the building. After block 425, the routine continues to block 435 to optionally preprocess the acquired 360° spherical panorama images before their subsequent use for generating related mapping information, such as to perform an equirectangular projection for each such image…).
Pham also teaches wherein the using of the at least revised predicted amount of time includes providing revised feedback to the one or more users that includes the revised predicted amount of time, to enable further improvement in producing the floor plan(Fig. 12 and Abstract, The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions… section III, If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate[20], [25]. To address this problem, we propose a fast adaptive selection algorithm to eliminate unnecessary input images that are densely overlapped with other images). On figure 12, it also shows the amount of time it takes was less after the above improvement on the proposed method.
As of claim 16, the modified model teaches all the limitation of claim 15, and Li also teaches wherein the generating of the floor plan for the building further includes obtaining input from one or more users related to at least one of the final room shapes for the multiple rooms (para 45, After all of the room shape layout information has been specified and any such wall width information has been determined, the final results may be used to generate a 20 floor map of the house… [68] Once the user is done, the final room shape in the second GUI portion provides a user-defined estimate of the room shape. As noted elsewhere herein, in some embodiments the routine may further perform one or more types of automated modifications to the user defined room shape estimate to determine the final defined room shape, while in other embodiments the user-defined room shape estimate may be used as the final defined room shape,
Figure 5A label 535…[79] a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building)
or the combining of the final room shapes, and using the obtained input as part of the generating of the floor plan.
As of claim 18, the modified model teaches all the limitations of claim 14, and Li also teaches wherein the first and additional images are each a panorama image having 360 degrees of horizontal visual coverage around a vertical axis, (para
20, Figure 1 B depicts a block diagram of an exemplary building interior environment in which 360° spherical panorama images are generated, for use by the MIGM system to generate and provide a corresponding building floor map, as discussed in greater detail with respect to Figures 2A-2W, as well as for use in presenting the panorama images to users. In particular, Figure 1 B illustrates one story of a multi-story building 198 with an interior that was captured at least in part via multiple panorama images, such as by a mobile image acquisition device 185 with image acquisition capabilities as it is moved through the building interior to a sequence of multiple viewing locations 210 (e.g., starting at viewing location 210A, moving to viewing location 21 OB along travel path 115, etc.).
wherein acquiring of the first and additional images is performed without using any depth information from any depth-sensing sensors for distances to surrounding surfaces(para 09 … analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building ….in at least some such embodiments, the generating of the mapping information is further performed without having or using information acquired from depth-sensing equipment about distances from the images' viewing locations to walls or other objects in the surrounding building interior)and includes using one or more inertial measurement unit (IMU) sensors of the image acquisition computing device to acquire motion data, and (para
20, While such a mobile image acquisition device may include various hardware components, such as a camera, one or more sensors (e.g., a gyroscope, an accelerometer, a compass, etc., such as part of one or more IMUs, or inertial measurement units, of the mobile device; an altimeter; light detector; etc.), a GPS receiver, one or more hardware processors, memory, a display, a microphone, etc.,
label 422 Optionally obtain acceleration data and/or other linking information generated by image acquisition device as it moves to next viewing location in sequence)
wherein the stored instructions include software instructions that, when
executed by the one or more hardware processors, cause the image acquisition
computing device to perform further automated operations including: (para 54, In the illustrated embodiment, an embodiment of the ICA system 340 executes in memory 330 of the server computing system(s) 300 in order to perform at least some of the described techniques, such as by using the processor(s) 305 to execute software instructions of the system 340 in a manner that configures the processor(s) 305 and computing system 300 to perform automated operations that implement those described techniques).
determining, for each of the first and additional images, acquisition pose information based at least in part on the motion data acquired by the one or more
IMU sensors during acquiring of that image and on visual data of that image, and (para 16, The illustrated embodiment of mobile device 185 further includes one or
more sensor modules 148 that include a gyroscope 148a, accelerometer 148b
and compass 148c in this example (e.g., as part of one or more IMU units, not
shown separately, on the mobile device), optionally a GPS (or Global Positioning
System) sensor or other position determination sensor (not shown in this
example), a display system 142, etc. [72] The image and the specified horizontal visual indicators are then further analyzed to determine the height of the camera within the room when the image was taken, such as based on the angle between horizontal visual indicators on two adjacent walls, [73] … The image and the specified vertical visual indicators are then further analyzed to triangulate the location of the camera within the room when the image was taken, such as based on the distance in the image between vertical visual indicators relative to their locations on the walls of the room shape).
the one or more first images is based in part on the respective determined acquisition pose information for each of the one or more first images ( para 70 -71, After block 530, the routine continues to block 535 to determine if there is another room (or another 360° spherical panorama image with an equirectangular projection for a room) for which to determine a room shape and additional related information, and if so returns to block 520 … In some embodiments, the routine may further perform automated operations to identify a candidate position of the image (such as a likely position from image analysis and using machine learning techniques), and then use that information to guide the image position determination).
As of claim 20, the modified model teaches all the limitations of claim 10, and Li also teaches wherein the analyzing of the visual data of the one or more first images includes identifying one or more problems to correct that include at least one of a lack of inter-image line-of-sight between visual data of at least one of the first images and at least one other acquired image,
or a lack of overlap between visual data of at least one of the first images and at least one other acquired image,
or a quantity of acquired images that is below a defined minimum threshold, (para
62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality, or do not appear to provide complete coverage of the building.
or a quantity of acquired images that is above a defined maximum threshold,
or a distance between an acquisition location of one of the first images and at least one wall, or a lack of line-of-sight through at least one doorway for at least one of the first images, or visibility of at least a portion of equipment used in acquiring at least one of the first images, or changes to the equipment used in acquiring at least one of the first images relative to acquiring one or more other images, or visibility of at least a portion of a user involved in acquiring at least one of the first images, or visibility of obstructions in visual data of at least one of the first images that block visibility of one or more walls, or a lack of coverage of all of the multiple rooms, or a lack of coverage of all of at least one of the multiple rooms, or uncertainty in determination of a room shape of at least one of the multiple rooms based on visual data of at least one of the first images,
or an inability to detect one or more windows or doorways or non-doorway wall openings in visual data of at least one of the first images, and
wherein the providing of the feedback further includes providing information to correct the identified one or more problems( para 40, As illustrated in the example of the seventh GUI pane of Figure 20, however, the initial relative layout of the room shapes may not be precisely aligned at first, such as with room shape 239 being at a slight angle relative to that of room 238 (e.g., due to imperfections in the locations and/or connections of the inter-room openings). While such layout problems may be immediately adjusted by the user and/or by the MIGM system in an automated manner (e.g., by applying layout constraints and/or optimizations, such as that room shapes connect at parallel walls) in some embodiments and situations
[62] After block 425, the routine continues to block 435 to optionally preprocess the acquired 360° spherical panorama images before their subsequent use for generating related mapping information, such as to perform an equirectangular projection for each such image, such that straight vertical data (e.g., the sides of a typical rectangular door frame, a typical border between 2 adjacent walls).
As of claim 21, the modified model teaches all the limitations of claim 10, and Li also teaches wherein generating of predicted and revised predicted amounts of time includes using information about attributes of images acquired for the producing of the mapping information for the building in combination with additional information about at least one of a size of the building,(para 12 As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations, other audio information, such as recordings of ambient noise; overall dimension information, etc. …. As another example, in at least some embodiments, additional processing of images is performed to determine estimated distance information of one or more types, such as to measure sizes in images of objects of known size, and use such information to estimate room width, length and/or height dimensions. Such estimated size information for one or more rooms may be associated with the floor map).or a style of the building, or a number of stories of the building, or a number of levels of the building, or a number of rooms of the building, or a number of rooms of an indicated type of the building, or a type of one or more rooms of the building, or a type of one or more walls of the building, or whether the building is furnished, or data available from one or more sensors of one or more types on one or more image acquisition devices used for acquiring the images for the producing of the mapping information for the building, or a quantity of the images acquired for the producing of the mapping information for the building, or an assessed quality of the images acquired for the producing of the mapping information for the building, or an amount of coverage of all of the multiple rooms by visual data of the images acquired for the producing of the mapping information for the building, or uncertainty in determination of room shapes of the multiple rooms based on visual data of the images acquired for the producing of the mapping information for the building, or detection of at least one of windows or doorways or non-doorway wall openings in visual data of the images acquired for the producing of the mapping information for the building, or an amount of computing resources available for the producing of the mapping information for the building, or availability of one or more users to assist in the producing of the mapping information, or information about at least one user involved in acquiring the images for the producing of the mapping information for the building, or about one or more other buildings having similarities to the building.
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), in the view of Sensopia Inc.; "Magicplan - 2D/3D Floor Plans on the App Store";2019; pgs. 1-2, further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages.
As of claim 2, the combined model of Li and Pham teaches all the limitations of claim1, and Li also teaches wherein the providing of the feedback and of the further feedback includes displaying the feedback and the further feedback in a graphical user interface shown on the computing device( fig 5A, label 520 optionally automatically determine likely room corners and/ or borders, and display GUI that includes a first portion with the image and a user-manipulatable floor-wall, ceiling-wall and wall-wall room outline and optionally indications of likely corners and/or borders and that includes a second portion with an associated initial room shape) where the method further comprises: before the repeated updating, generating, by the computing device and for each of the one or more first rooms, an initial estimate of a room shape for that first room using the first visual data, and displaying, together with the feedback to the user, a visual representation for each of the one or more first rooms of the generated initial estimate of the room shape for that first room; (para 29-30, The second GUI pane 255e displays an initial room shape 260e corresponding to the visual border and corner GUI controls 280 and 285 in the first GUI pane, such that changes to the room shape in the second GUI pane or to the visual border and corner GUI controls 280 and 285 in the first GUI pane cause corresponding changes in the other pane… After the corner and border visual representations are displayed in the GUI, the user is able to manipulate the visual representations to match corresponding features of the room that are visible in the underlying panorama image, as illustrated in further detail with respect to Figure 2F (feedback)).
for each of the multiple iterations, generating, by the computing device and
for each of the at least one second rooms at which the at least one second
panorama image for that iteration is acquired, an initial estimate of a room shape for that at least one second room using the second visual data,(para 67, and figure 5A (Room shape determination), In block 520, the routine receives a selection (e.g., from an operator user of the MIGM system) of a 360° spherical panorama image with an equirectangular projection taken in a room of the building and/or of a room of the building having such a 360° spherical panorama image with an equirectangular projection, and proceeds to use the corresponding panorama image to define a shape of the room – as discussed in greater detail elsewhere herein, in situations in which a room has multiple 360° spherical panorama images, only one image may be analyzed in the described manner in some embodiments, or instead other of the additional images may be subsequently analyzed in other embodiments (e.g., to generate a refined version of the room shape based on a combination of information from multiple such analyzed images) displaying, together with the further feedback to the user, a further visual representation for each of the at least one second rooms of the generated initial estimate of the room shape for that at least one second room ( para 67, In some embodiments, the routine may further perform automated operations to identify candidate locations of borders (e.g., between adjacent walls, between walls and a ceiling, and/or between walls and a floor) and/or corners in the room (such as via image analysis and using machine learning techniques), and then use that information to guide the room shape determination. The routine proceeds in block 520 to display a GUI (or update a previously displayed GUI) to show the panorama image in a first pane or other first portion of the GUI, while including a second pane or other second portion to show a visual representation of the changing room shape as it is being incrementally define).
The combined model does not explicitly teach the predicted amount of time to complete generation of the floor plan is based on initial room shape uncertainties for the first and second rooms.
While Magicplan teaches wherein generating of the predicted amount of time to complete the generation of the floor plan for the building is (section, screenshot, Create floor plans simply with the camera of your mobile device. Scan a room in up to 30 seconds and build up complete floor plans in minutes. A floor plan is your starting point for included features like material & cost estimation, 3D models, and virtual tours).
Magicplan is considered to be analogous to the claimed invention, since it focus on floor plan generation. Therefore a person of ordinary skill in the art before the effective filling date can apply Magicplan teaching amount of time to generate a floor plan to the combined model in order to create floor plans with the camera of your mobile device in a minute(Magicplan)
The modified model doesn’t teach effect of room uncertainty on amount of time while CubiCasa teaches based at least in part on first room shape uncertainties associated with the generated initial estimates of the room shapes for each of the first rooms, and on second room shape uncertainties associated with the generated initial estimates of the room shapes for each of the at least one second rooms for that iteration(FAQs, 2o How long it takes to take a video from a 2000 sq ft home which 1s in 2 floors? Capturing process takes from 4-6, minutes in 2000sq ft space . capturing time depends on the area and complexity of the indoor space. 41. How long does H: take to build a floorplan of a 4-bed house? The scanning speed can be fairly fast, so 1000 sqft should not take more than 5 minutes.) as listed above room shape uncertainty is mapped to the area and complexity of the indoor space and size.
Cubicasa is considered to be analogous to the claimed invention since it focus on floor plan generation. Therefore it would be obvious to a person of ordinary skill in the art before the effective filling date to combine Cubicasa teaching of providing updated amount of time it takes based on space and complicity for one room and 4 room, to the modified model in order to increase the accuracy of generation process with the error margin of around 5%(Cubicasa, FAQ).
As of claim 3 the combined model teaches all the limitations of claim 2, and Li also teaches transmitting, by the computing device and to one or more server computing systems over one or more computer networks, the one or more first panorama images (fig 3 from label 300 and 360 to 390 by using a network 399)
for each of the multiple iterations, the one or more second panorama images acquired for that iteration;( fig 4, label 435 Optionally preprocess acquired images before further analysis, and the process is iterative as shown after label 495, it goes back to 405, so it continues acquiring images)
performing, by the one or more server computing systems, the automated
floor plan generation process using the first visual data and using the second
visual data of the one or more second panorama images acquired for each of the
multiple iterations(abstract, The defined area may include an interior of a multi-room building, and the generated information including a floor map of the building, such as from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees around a vertical axis), [79], a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building) including to determine final room shapes for the multiple rooms and to generate the floor plan for the building using the final room shapes;( fig 5A , labeled 530 optionally receive user annotations or other non-image information to link to room or to locations within room and add corresponding visual indicators, optionally apply automated optimization(s) to room shape based on geometrical constraints and/or other information, optionally determine room dimensions from one or more known or estimated lengths, and store final room shape along with other determined/received information)
transmitting, by the one or more server computing systems and to the
computing device over the one or more computer networks, the generated floor
plan for the building (claim 7, wherein the presenting of the generated building floor map includes transmitting, by the one or more computing devices, the generated floor map to one or more client devices of the one or more additional users for display on the one or more client devices).
displaying, by the computing device and to the user, the generated floor
plan for the building (para 09, The generated floor map and/or other generated mapping-related information may be further used in one or more manners in various embodiments, such as for controlling navigation of mobile devices (e.g., autonomous vehicles), for display on one or more client devices in corresponding GUls (graphical user interfaces), etc.).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), in the view of Sensopia Inc.; "Magicplan - 2D/3D Floor Plans on the App Store";2019; pgs. 1-2, further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages,further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013.
As of claim 4, the modified model teaches all the limitations of claim 3, and Li also teaches before the acquiring of the one or more first panorama images,
determining, by the computing device, computing resources available to the server computing system at a current time for the automated floor plan generation process; (para 56 As one non-limiting example, the mobile image acquisition devices 360 are each shown to include one or more hardware CPU(s) 361, 1/0 components 362, storage 365, and memory 367, with one or both of a browser and one or more client applications 368 (e.g., an application specific to the MIGM system and/or ICA system)
The combined model does not explicitly teaches presenting, by the computing device, an initial prediction of an amount of time to complete the generation of the floor plan for the building that is determined based at least in part on the determined computing resources and on information about past performance of the user in acquiring images and on publicly available information about the building, and wherein the generating of the predicted amount of time to complete the generation of the floor plan for the building and the generating of the revised predicted amount of time for each of the multiple iterations is further based in part on the determined computing resources.
While Williams teaches presenting, by the computing device, an initial prediction of an amount of time to complete the generation of the floor plan for the building that is determined based at least in part on the determined computing resources and on information about past performance of the user in acquiring images and on publicly available information about the building (page 129 Computer platform – Image stitching always involves large final images (1-3 Gbytes) as well as relatively large tiles (150-250 Mbytes). A typical number of tiles per image is between four and twelve and sometimes as large as thirty. To make the processing of such large files feasible the computer platform and associated drives need to have sufficient memory, swap disk space, and often, multi-core capability. Indeed, some stitching programs monitor the progress of the stitching process and make dynamic decisions on how to best provide a final image in a reasonable amount of time. If computer resources become scarce, certain logical, adaptive, decisions in the algorithm are made. The result is often a suboptimal stitch. This sometimes manifests itself with variable results at different execution times (e.g., day 1 vs. day 2) despite providing the software the exact same image files and processing selections)
wherein the generating of the predicted amount of time to complete the generation of the floor plan for the building and the generating of the revised predicted amount of time for each of the multiple iterations is further based in part on the determined computing resources (page 129, If computer resources become scarce, certain logical, adaptive, decisions in the algorithm are made. The result is often a suboptimal stitch. This sometimes manifests itself with variable results at different execution times (e.g., day 1 vs. day 2) despite providing the software the exact same image files and processing selections. It is for this reason that high-speed solid-state drives (SSD) are often recommended to be installed on such platforms where stitching operations are done).
Williams is considered to be analogous to the claimed invention since it teaches combining of images to create a complete scene. Therefore it would be obvious to try by a person of ordinary kill in the art before the effective filling date to use Williams teaching of computational resource used for image stitching and the recommended resources (feedback) for optimal stitch on to the combined model to perform generation process.
The motivation would have been Once we have a basic understanding of the steps used by current image stitching software, we can make several suggestions
that will improve results. The first is to control the lighting and relative position of the component tile images so that they are approximately related by simple translation across the object. The second is to characterize the camera taking lens distortion (e.g. barrel or pincushion), and correct each image tile prior to image stitching( Williams, conclusion)
Claims 9 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021) further in the view of Fathi; Habib (US 11106911 B1).
As of claim 9, the modified model teaches all the limitations of claim 8, and Pham also teaches predicting, by the one or more computing devices and based at least in part on one or more assessed attributes of the one first image, an adjusted predicted amount of time to complete the floor plan for the building that is less than the predicted amount of time based at least in part on the one or more improvements; ( Section III, If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate [20], [25]. To address this problem, we propose a fast adaptive selection algorithm to eliminate unnecessary input images that are densely overlapped with other images. Accordingly, feature matching and registration processes are only performed in the overlapping regions of the correlated images. Thus, in the registration process, the proposed method can considerably reduce the number of outliers) On figure 12, it also shows the amount of time it takes was less after the above improvement on the proposed method and Pham also teaches conducted extensive experiments to prove the effectiveness of the proposed method in reducing the processing time.
displaying, by the image acquisition computing device, the adjusted
predicted amount of time (figure 12, display as a graph , section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively).
The combined model does not explicitly teach acquiring, by the image acquisition computing device after the providing of the feedback, a new image to replace the one first image that includes one or more improvements to correct the one first image
While Fathi teaches acquiring, by the image acquisition computing device after the providing of the feedback, a new image to replace the one first image that includes one or more improvements to correct the one first image (para 63, (63) Such identification of one or more problematic image acquisition aspects or characteristics that occurred in a first image acquisition event and the reduction or elimination of such problematic aspects in a second image acquisition event can be generated automatically by creating a new capture plan for the second (or subsequent) image acquisition event or by modification of the first capture plan to include corrections as indicated from failure analysis of the output of the first image acquisition event. In this regard, correction of the second capture plan can be provided substantially without user interaction).
Fathi is considered to be analogous to the claimed invention since it focus on Image Acquisition Planning Systems. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date to integrate Fathi teaching of acquiring a new image to replace the problematic image on the modified model of floor plan generation process.
The motivation would have been capturing a second image to replace problematic images improve the accuracy and image acquisition process, it could be expected that the 3D information derivable from an image acquisition event be improved over time (Faith [62] –[66]).
Claim 17 is also in the same scope to claim 9, so claim 17, is also rejected under the same rational as of claim 9.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Artiano, JR (US20200257832 A1).
As of claim 13, the combined model teaches all the limitations of claim 12, and Li also teaches wherein the providing of the feedback and the revised feedback to the one or more users includes displaying the feedback and the revised feedback in a graphical user interface, (para 72, The user also moves the corresponding wall visual indicators in the seventh GUI portion to show those walls on which the corresponding visual horizontal indicators in the sixth GUI portion indicate horizontal spaces. As the user manipulates the visual horizontal indicators in the sixth GUI portion and/or the wall visual indicators in the room shape in the seventh GUI portion, the information in both GUI portions is updated to reflect the user manipulations (e.g., simultaneously with the user manipulations) and show the user changes that are made. The image and the specified horizontal visual indicators are then further analyzed to determine the height of the camera within the room when the image was taken, such as based on the angle between horizontal visual indicators on two adjacent walls).
and wherein the providing of the floor plan for the building is performed after the floor plan is completed (para 12, Such estimated size information for one or more rooms may be associated with the floor map, stored and optionally displayed - if the size information is generated for all rooms within a sufficient degree of accuracy, a more detailed floor plan of the building may further be generated, such as with sufficient detail to allow blueprints or other architectural plans to be generated).
The combined model does not explicitly teach displaying the completed floor plan in the graphical user interface.
While Artiano teaches displaying the completed floor plan in the graphical user interface (para, 04, Generating the floor plan may include generating and displaying, on the graphical user interface, a navigable three-dimensional model of the structure based upon, at least in part, the scanning of the one or more portions of the structure)
Artiano is considered to be analogous to the claimed invention since it focuses on floor plan generation. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date, to combine Artiano teaching of displaying the floor plan on GUI on to the modified model.
The motivation would have been by displaying the floor plan to the user the labeled floor plan and/or rendered model may provide emergency services or other third parties with an objective reference system for communicating and identifying the location of various portions of a structure or building (Artiano, para 35)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of GANIHAR (US 20210232719 A1).
As of claim 19, the modified model teaches all the limitations of claim 10, and Li also teaches wherein the automated operations further include presenting the mapping information to enable interactions with the presented mapping information by one or more users( para, 10 In addition, the automated operations of the computing device(s) may in some embodiments and situations include interacting with one or more MIGM system operator users who assist with the analysis of the images and the generating of the mapping information, such as by displaying one or more GUls that show information related to the images and/or that show associated mapping information being generated, and receiving and further using input submitted by the user(s) via the GUl(s) as part of the mapping information generation).
The combined model does not explicitly teach wherein the mapping information includes at least one of a three-dimensional model of an interior of the building.
While Ganihar teaches wherein the mapping information includes at least one of a three-dimensional model of an interior of the building (para 08, It is a general object of the present disclosure to provide system and method for generating and visualizing an interior design pertaining to a floor plan in a three-dimensional representation)
Ganihar is considered to be analogous to the claimed invention since it focus on interior design pertaining to a floor plan. Therefore it would be obvious for a person of ordinary skill in the art, before the effective filling date to integrate Ganihar teaching of interior 3D model in to the modified model.
The motivation would have been it would be appreciated that, creation of 3D representation of the interior design allows a user to visualize placement of various objects such as doors, windows, lighting, furniture, fixtures, etc in various spaces of a floor plan (Ganihar, [108]).
Claims 22, 24, 26- 29 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages.
As of claim 22, the modified model teaches all the limitation of claim 10, while Pham teaches wherein acquiring of the first and additional images for the building is performed by one or more users (para 17, … a camera having sufficient fisheye lenses to capture 360 degrees horizontally without rotation, a smart phone held and moved by a user, a camera held by or mounted on a user or the user's clothing, etc.) to capture data from a sequence of multiple viewing locations within multiple rooms of a house (or other building) … the techniques may include producing a 360° spherical panorama image from that viewing location).wherein the using of the at least revised predicted amount of time includes providing further feedback that includes at least the revised predicted amount of time(
PNG
media_image1.png
567
711
media_image1.png
Greyscale
As shown above on the graph , for different image number it have updated amount of time it takes for stitching using different model and the proposed model use image attributes like number of images and resolution. (Section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively), As shown on Figure, the revised amount of time it takes would be the amount of time it takes for the proposed method since the proposed method make improvements from the other models.
and wherein the automated operations further include, before acquiring any images of the building for use in the producing of the mapping information for the building: determine in computing resources available for implementing the generation process(Section c, “Experimental result”, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM and an NVIDIA GeForce RTX 2070 GPU).
and presenting to the one or more users the determined initial prediction of the amount of time, (Figure 12, the amount of time it takes is shown or the user using graph).
The combined model does not explicitly teach determining an initial prediction of an amount of time for the producing of the mapping information for the building that is based at least in part on the determined computing resources and on information about the one or more users and on publicly available information about the building; and wherein the providing of the feedback and of the revised feedback is performed to provide repeated updates to the one or more users of a predicted amount of time based on cumulative acquisition of images for the building.
Williams teaches determining an initial prediction of an amount of time for the producing of the mapping information for the building that is based at least in part on the determined computing resources and on information about the one or more users and on publicly available information about the building; (page 129 Computer platform – Image stitching always involves large final images (1-3 Gbytes) as well as relatively large tiles (150-250 Mbytes). A typical number of tiles per image is between four and twelve and sometimes as large as thirty. To make the processing of such large files feasible the computer platform and associated drives need to have sufficient memory, swap disk space, and often, multi-core capability. Indeed, some stitching programs monitor the progress of the stitching process and make dynamic decisions on how to best provide a final image in a reasonable amount of time. If computer resources become scarce, certain logical, adaptive, decisions in the algorithm are made. The result is often a suboptimal stitch. This sometimes manifests itself with variable results at different execution times (e.g., day 1 vs. day 2) despite providing the software the exact same image files and processing selections).
Williams is considered to be analogous to the claimed invention since it teaches combining of images to create a complete scene. Therefore it would be obvious to try by a person of ordinary kill in the art before the effective filling date to use Williams teaching of computational resource used for image stitching and the recommended resources (feedback) for optimal stitch on to the combined model to perform generation process.
The motivation would have been Once we have a basic understanding of the steps used by current image stitching software, we can make several suggestions
that will improve results. The first is to control the lighting and relative position of the component tile images so that they are approximately related by simple translation across the object. The second is to characterize the camera taking lens distortion (e.g. barrel or pincushion), and correct each image tile prior to image stitching( Williams, conclusion)
The modified model does not explicitly teach wherein the providing of the feedback and of the revised feedback is performed to provide repeated updates to the one or more users of a predicted amount of time based on cumulative acquisition of images for the building.
While Cubicasa teaches wherein the providing of the feedback and of the revised feedback is performed to provide repeated updates to the one or more users of a predicted amount of time based on cumulative acquisition of images for the building (FAQ, 8, 8. How much should I scan? Scanning time depends on the indoor space and complexity. You should be able to capture one room fairly fast, in 15-20 seconds. 41. How long does H: take to build a floorplan of a 4-bed house? The scanning speed can be fairly fast, so 1000 sqft should not take more than 5 minutes).
Cubicasa is considered to be analogous to the claimed invention since it focus on floor plan generation. Therefore it would be obvious to a person of ordinary skill in the art before the effective filling date to combine Cubicasa teaching of providing updated amount of time it takes based on space and complicity for one room and 4 room, to the modified model in order to increase the accuracy of generation process with the error margin of around 5%(Cubicasa, FAQ).
As of claim 24, Li teaches A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations including at least:(Para 58, Some or all of the components, systems and data structures may also be stored (e.g., as software instructions or structured data) on a non-transitory computer-readable storage mediums, such as a hard disk or flash drive or other non-volatile storage device, volatile or non-volatile memory (e.g., RAM or flash RAM), a network storage device, or a portable media article (e.g., a DVD disk, a CD disk, an optical disk, a flash memory device, etc.) to be read by an appropriate drive or via an appropriate connection).
obtaining, by the one or more computing devices and for a building with multiple rooms, one or more first images acquired by one or more users at one or more first acquisition locations in one or more first rooms(para 09, In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 2D (two- dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 3600 spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360).
wherein each of the first images has visual coverage of at least some walls of one of the first rooms that includes the respective first acquisition location for that first image; (para 25, In particular, Figure 2A illustrates an example image 250a, such as a perspective image taken in a northeasterly direction from viewing location 210B in the living room of house 198 of Figure 1B (or a northeasterly facing subset view of a 360-degree panorama image taken from that viewing location and formatted in a rectilinear manner)…. In the illustrated example, the displayed image includes built-in elements (e.g., light fixture 130a), furniture (e.g., chair 192- 1 ), two windows 196-1, a picture 194-1 hanging on the north wall of the living room. No inter-room passages into or out of the living room (e.g., doors or other wall openings) are visible in this image).
analyzing, by the one or more computing devices, visual data of the one or more first images to assess one or more attributes of the one or more first images;(Para 75, In some embodiments, the routine may further perform automated operations to identify candidate locations of each inter-room passage between the first and second rooms in both of the first and second images (such as likely locations
from image analysis and using machine learning techniques), and then uses that information to guide the layout determination of the room shapes for the first and second rooms).
wherein the automated generation process is based on analysis of the visual data of the one or more first images and is based on future analysis of additional images to be acquired at additional acquisition locations for the building,(para 12, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations, other audio information…)
obtaining, by the one or more computing devices, one or more second images of the additional images that are acquired by the one or more users at one or more additional second acquisition locations for the building;(para 61, After block 415 is completed, the routine continues to block 420 to determine if there are more viewing locations at which to acquire images, such as based on corresponding information provided by the user of the mobile device. If so, and when the user is ready to continue the process, the routine continues to block 422 to optionally initiate the capture of linking information (including acceleration data) during movement of the mobile device along a travel path away from the current viewing location and towards a next viewing location within the building interior).
analyzing, by the one or more computing devices, visual data of the one or more second images to assess at least one attribute of the one or more second
images;(para 75, In some embodiments, the routine may further perform automated operations to identify candidate locations of each inter-room passage between the first and second rooms in both of the first and second images (such as likely locations from image analysis and using machine learning techniques), and then uses that information to guide the layout determination of the room shapes for the first and second rooms).
Li does not explicitly teach wherein the predicted one or more characteristics include a predicted amount of one or more resources involved with the producing of the floor plan for the building; predicting, by the one or more computing devices and based at least in part on the assessed one or more attributes of the one or more first images, one or more characteristics of an automated generation process to produce a floor plan for the building, providing, by the one or more computing devices, feedback to the one or more users that includes at least the predicted amount of the one or more resources, to enable improvement in the automated generation process resulting at least in part from acquiring of the additional images, predicting, by the one or more computing devices and based at least in part on the assessed at least one attribute of the one or more second images, one or more updated characteristics of the automated generation process that include a revised predicted amount of the one or more resources involved with the producing of the floor plan for the building; and providing, by the one or more computing devices, revised feedback to the one or more users that includes at least the revised predicted amount of the one or more resources, to enable further improvement in the automated generation process.
While Pham teaches predicting, by the one or more computing devices and based at least in part on the assessed one or more attributes of the one or more first images, one or more characteristics of an automated generation process to produce a floor plan for the building, (Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Section 4 “processing time”, As shown in Figure 12, we evaluate the performance of the proposed method and five cutting-edge stitching software, in terms of the total processing time for the entire stitching process with the number of images of the Technology Park scene varying from 10 to 200. The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically, especially for Pix4d [35], Hugin [42], and Photoshop [43]. The pre-processing time of the proposed method is also added for a fair comparison. The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively)
providing, by the one or more computing devices, feedback to the one or
more users that includes at least the predicted amount of the one or more
resources, to enable improvement in the automated generation process resulting
at least in part from acquiring of the additional images;(Abstract The proposed method improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods, Section iv , c, The simulations were performed on a Windows 10 computer equipped with an Intel Core i7-10700F CPU, 32GB RAM, Figure 12 show feedback including time , for resolution of and number of images).
Li and Pham are considered to be analogous to the claimed invention since they focus on automatic operation of involved image analysis and combining images to create a complete scene. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to combine Pham teaching of estimating amount of time based on different attributes in to Li ‘s model in order to further provide benefits in allowing improved automated navigation of a building by mobile devices (e.g., semi-autonomous or fully autonomous vehicles), including to significantly reduce their computing power used and time used to attempt to otherwise learn a building's layout (Li, para 13) and improves the visual quality of the stitched image, by decreasing the estimated reprojection error and the number of observed visual distortions. In addition, the proposed method can substantially reduce the processing time compared with conventional stitching methods( Pham, abstract).
The modified model does not explicitly teach wherein the predicted one or more characteristics include a predicted amount of one or more resources involved with the producing of the floor plan for the building; predicting, by the one or more computing devices and based at least in part on the assessed at least one attribute of the one or more second images, one or more updated characteristics of the automated generation process that include a revised predicted amount of the one or more resources involved with the producing of the floor plan for the building; and providing, by the one or more computing devices, revised feedback to the one or more users that includes at least the revised predicted amount of the one or more resources, to enable further improvement in the automated generation process.
Williams teaches predicting, by the one or more computing devices and based at least in part on the assessed at least one attribute of the one or more second images, one or more updated characteristics of the automated generation process that include a revised predicted amount of the one or more resources involved with the producing of the floor plan for the building (page 129 Computer platform – Image stitching always involves large final images (1-3 Gbytes) as well as relatively large tiles (150-250 Mbytes). A typical number of tiles per image is between four and twelve and sometimes as large as thirty. To make the processing of such large files feasible the computer platform and associated drives need to have sufficient memory, swap disk space, and often, multi-core capability. Indeed, some stitching programs monitor the progress of the stitching process and make dynamic decisions on how to best provide a final image in a reasonable amount of time. If computer resources become scarce, certain logical, adaptive, decisions in the algorithm are made. The result is often a suboptimal stitch. This sometimes manifests itself with variable results at different execution times (e.g., day 1 vs. day 2) despite providing the software the exact same image files and processing selections)
providing, by the one or more computing devices, revised feedback to the one or more users that includes at least the revised predicted amount of the one or more resources, to enable further improvement in the automated generation process (page 129, If computer resources become scarce, certain logical, adaptive, decisions in the algorithm are made. The result is often a suboptimal stitch. This sometimes manifests itself with variable results at different execution times (e.g., day 1 vs. day 2) despite providing the software the exact same image files and processing selections. It is for this reason that high-speed solid-state drives (SSD) are often recommended to be installed on such platforms where stitching operations are done).
Williams is considered to be analogous to the claimed invention since it teaches combining of images to create a complete scene. Therefore it would be obvious to try by a person of ordinary kill in the art before the effective filling date to use Williams teaching of computational resource used for image stitching and the recommended resources (feedback) for optimal stitch on to the combined model to perform generation process.
The motivation would have been Once we have a basic understanding of the steps used by current image stitching software, we can make several suggestions
that will improve results. The first is to control the lighting and relative position of the component tile images so that they are approximately related by simple translation across the object. The second is to characterize the camera taking lens distortion (e.g. barrel or pincushion), and correct each image tile prior to image stitching( Williams, conclusion)
The modified model does not explicitly teach wherein the predicted one or more characteristics include a predicted amount of one or more resources involved with the producing of the floor plan for the building.
While , Cubicasa teaches wherein the predicted one or more characteristics include a predicted amount of one or more resources involved with the producing of the floor plan for the building (FAQ 28. How much this consumes my phone battery? Scanning requires a lot of processing power, but you should not lose more than 5% of battery even with a larger property).
Cubicasa is considered to be analogous to the claimed invention since it focus on floor plan generation. Therefore it would be obvious to a person of ordinary skill in the art before the effective filling date to combine Cubicasa teaching of providing updated amount of resource it takes on the modified model in order to increase the accuracy of generation process with the error margin of around 5% based on space and complicity for one room and 4 room Cubicasa, FAQ).
As of claim 26, the modified model teaches all the limitations of claim 24, and Cubicasa also teaches wherein the predicted amount of resources involved with the producing of the floor plan for the building includes at least one of an amount of time until completion of the producing of the floor plan, or an amount of computing resources used in the producing of the floor plan, ( FAQ 28. How much this consumes my phone battery? Scanning requires a lot of processing power, but you should not lose more than 5% of battery even with a larger property.
or an amount with respect to one or more indicated cost metrics.
As of claim 27, the combined model teaches all the limitations of claim 24, and Li also teaches wherein the one or more computing devices include an image acquisition computing device with one or more cameras that is used to acquire the one or more first images and the additional images, (para 09, In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 20 (two dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis), …[20] While such a mobile image acquisition device may include various hardware components, such as a camera, one or more sensors (e.g., a gyroscope, an accelerometer, a compass, etc.)
wherein the analyzing of the visual data of the one or more first images is performed by the image acquisition computing device and includes (para 09…floor map that is generated from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having one or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis).
determining, for each of one or more first rooms that include the first acquisition locations, an initial estimated room shape of that first room, and wherein the providing of the feedback to the one or more users is performed by the image acquisition computing device(para 11, As one example, one or more types of additional information about a building may be received and associated with the floor map (e.g., with particular locations in the floor map), such as additional images, textual and/or audio annotations or other descriptions of particular rooms or other locations,
[30] After the corner and border visual representations are displayed in the GUI, the user is able to manipulate the visual representations to match corresponding features of the room that are visible in the underlying panorama image, as illustrated in further detail with respect to Figure 2F.(feedback)
[46] As in Figures 2E-2H, a second GUI pane 255s is shown with an initial room shape 260s corresponding to the visual border and corner GUI controls 280, 282 and 285 in the first GUI pane, Label 581 Receive selection of room with defined room shape and image from within room, optionally automatically determine likely image location in room from image analysis and/or metadata, and display GUI with image in sixth GUI portion and room shape in seventh GUI portion)
includes displaying information on the image acquisition computing device (Fig. 1A, display system 142, para 82, including to display images (e.g., 360° spherical panorama images) and/or other information associated with particular locations in the mapping information).and the initial estimated room shape for each of the one or more first rooms (para 29, The second GUI pane 255e displays an initial room shape...)and one or more indications of aspects of the one or more first images to change during acquisition of the additional images to produce the improvement in the additional images (para 62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality).
Pham also teaches displaying the predicted amount of time ( Figure 12) see claim 5 above so it would be obvious for a person of ordinary skill in the art to combine Li and Pham teaching to display the information including amount of time, room shape and aspect to change to produce the improvement in additional images.
As of claim 28, the combined model teaches all the limitations of claim 27 , and Li also teaches wherein the one or more computing devices further include one or more additional computing devices, and wherein the method further comprises (para
15 Figure 1A is an example block diagram of various computing devices and
systems that may participate in the described techniques in some embodiments. In particular, one or more 360° spherical panorama images 165 in equirectangular format have been generated by an Interior Capture and Analysis ("ICA") system (e.g., a system 160 that is executing on one or more server computing systems 180, and/or a system provided by application 155 executing on one or more mobile image acquisition devices 185),
transmitting, by the image acquisition computing device, data over one or
more networks to the one or more additional computing devices that includes
information from the one or more first images and the additional images;(para
53, The server computing system(s) 300 and executing ICA system 340, and server computing system(s) 380 and executing MIGM system 389, may communicate with each other and with other computing systems and devices in this illustrated embodiment via one or more networks 399 (e.g., the Internet, one or more cellular telephone networks, etc.), such as to interact with user client computing devices 390 (e.g., used to view floor maps, and optionally associated images and/or other related information) and on fig 3 from label 300 and 360 to 390 by using a network 399
generating, by the one or more additional computing devices and as part of
the automated generation process, the floor plan for the building using the
transmitted data, including determining final room shapes for the multiple rooms,
fig 5A label 530 optionally apply automated optimization(s) to room shape based on geometrical constraints and/or other information, optionally determine room dimensions from one or more known or estimated lengths, and store final room shape along with other determined/received information
[45], After all of the room shape layout information has been specified and any such wall width information has been determined, the final results may be used to generate a floor map of the house, [68] Once the user is done, the final room shape in the second GUI portion provides a user-defined estimate of the room shape.
[79] a floor plan may be generated from the floor map that includes dimension information for the rooms and the overall building, [claim 14] determining, by the one or more computing devices and for each of the one or more rooms, a final user-defined room shape for the room by combining information received from the user about borders of the room from each of the multiple images taken within the room.
and combining the final room shapes for the multiple rooms to complete the floor
plan; and (para 57, the routine continues to block 545 to determine if the instructions received in block 505 (or subsequently) indicate to combine room shapes of multiple indicated rooms in a building to determine their relative layout).
providing, by the one or more additional computing devices, the floor plan
for the building for further use (para 09, The generated floor map and/or other
generated mapping-related information may be further used in one or more
manners in various embodiments, such as for controlling navigation of mobile
devices (e.g., autonomous vehicles), for display on one or more client devices
in corresponding GUls (graphical user interfaces), etc.).
As of claim 29, the modified model teaches all the limitations of claim 27, and li also teaches wherein the providing of the feedback to the one or more users by the image acquisition computing device further includes displaying instructions to correct one of the first images based at least in part on at least one assessed attribute of the one first image that reflects quality of the visual data of the one first image (para 62, For example, the ICA system may provide one or more notifications to the user regarding the information acquired during capture of the multiple viewing locations and optionally corresponding linking information, such as if it determines that one or more segments of the recorded information are of insufficient or undesirable quality, or do not appear to provide complete coverage of the building. After block 425, the routine continues to block 435 to optionally preprocess the acquired 360° spherical panorama images before their subsequent use for generating related mapping information)
As of claim 31, the modified model teaches all the limitations of claim 24,
wherein the predicted one or more characteristics of the automated generation process for producing the floor plan for the building include a predicted amount of time to complete acquisition of images in all of the multiple rooms, and wherein the providing of the feedback to the one or more users further includes the predicted amount of time to complete the acquisition of the images in all of the multiple rooms (Cubicasa, on FAQ 2, capturing process takes from 4-6 minutes in 2000sq ft space, Capturing time depends on the area and complexity of the indoor space).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Bhogal; Nikhil (US 20120293607 A1)
As of claim 23, the modified model teaches all the limitations of claim 10, and Li teaches finding of first and second images iteratively and Pham also teaches wherein the using of the at least revised predicted amount of time includes providing further feedback that includes at least the revised predicted amount of time, and wherein the automated operations further include: (section 4 and Figure 12, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 * 3648 is 67.1, 106.5, and 135.7 (seconds), respectively) It provides the amount of time it takes after improvement was made to improve the image visual quality(conclusion).
wherein the providing of the feedback includes providing information to one or more users that includes the predicted amount of time (section 4, As shown in Figure 12, we evaluate the performance of the proposed method and five cutting-edge stitching software, in terms of the total processing time for the entire
stitching process with the number of images of the Technology Park scene varying from 10 to 200. The processing time of the proposed method increases linearly, whereas the time taken by other methods increases dramatically).
While combined model does not explicitly teach determining, based at least in part on the one or more first images and before acquiring of the one or more second images, a current status of acquiring images for the generation process that includes one or more characteristics of at least one of an amount completed of the acquiring of the images for the generation process or an amount remaining of the acquiring of the images for the generation process, the determined current status including the one or more characteristics; determining, based at least in part on the one or more second images, a revised current status of acquiring images for the generation process that includes revisions to the one or more characteristics, and wherein the providing of the further feedback includes providing information to the one or more users that includes the revised predicted amount of time and the determined revised current status.
Bhogal teaches determining, based at least in part on the one or more first images and before acquiring of the one or more second images, a current status of acquiring images for the generation process that includes one or more characteristics of at least one of an amount completed of the acquiring of the images for the generation process or an amount remaining of the acquiring of the images for the generation process, the determined current status including the one or more characteristics (para 72, (Step 1014). As each incoming lower-resolution image portion is stitched together with the growing panoramic image preview, the resultant panoramic preview image may be sent to the device's display to provide the user with a real-time or near real-time progress indicator for the panoramic image that is currently being captured (Step 1016). In some embodiments, the growing panoramic image preview may be overlaid on the device display with the scaled preview version of image referred to in Step 1004 above … [75], the panoramic photography process 200 described herein may provide the user with a seamless user experience including real-time progress feedback on the panoramic scene being captured, while simultaneously performing image stitching in substantially real-time--a feat previously thought to be too processing-intensive to achieve using handheld personal electronic devices).
determining, based at least in part on the one or more second images, a revised current status of acquiring images for the generation process that includes revisions to the one or more characteristics, and wherein the providing of the further feedback includes providing information to the one or more users that includes the revised predicted amount of time and the determined revised current status ( (para 72, (Step 1014). As each incoming lower-resolution image portion is stitched together with the growing panoramic image preview, the resultant panoramic preview image may be sent to the device's display to provide the user with a real-time or near real-time progress indicator for the panoramic image that is currently being captured (Step 1016). In some embodiments, the growing panoramic image preview may be overlaid on the device display with the scaled preview version of image referred to in Step 1004 above … [75], the panoramic photography process 200 described herein may provide the user with a seamless user experience including real-time progress feedback on the panoramic scene being captured, while simultaneously performing image stitching in substantially real-time--a feat previously thought to be too processing-intensive to achieve using handheld personal electronic devices). As it stated above the combined model already teaches iterative process and captures images, so the progress indicator will show the status for each image.
Bhogal is considered to be analogous to the claimed invention, because it focus on panorama image processing. Therefore it would be obvious to try by a person of ordinary skill in the art before the effective filling date Bhogal’s teaching of progress indicator in the generation process for the first and second images of the combined model.
The motivation would have been panoramic processing module may also provide feedback image registration information to allow the motion filter module to make more accurate decisions regarding correlating device positional movement to overlap amounts between successive image frames in the image stream. This feedback of information may allow more efficiently select image frames that will help image stitching.(Bhogal, para 84).
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages. Artiano, JR (US20200257832 A1).
As of claim 25, the modified model teaches all the limitations of claim 24, and Li also teaches wherein the one or more computing devices include an image acquisition computing device with one or more cameras that is used to acquire the one or more first images and the additional images,(para 09, In at least some embodiments, the defined area includes an interior of a multi-room building (e.g., a house, office, etc.), and the generated information includes a floor map of the building, such as a 20 (two dimensional) overhead view (e.g., an orthographic top view) of a schematic floor map that is generated from an analysis of multiple 360° spherical panorama images acquired at various viewing locations within the building (e.g., using an image acquisition device with a spherical camera having on or more fisheye lenses to capture a panorama image that extends 360 degrees horizontally around a vertical axis)…[20] While such a mobile image acquisition device may include various hardware components, such as a camera, one or more sensors (e.g., a gyroscope, a accelerometer, a compass, etc.). wherein the providing of the feedback and the revised feedback is performed by the image acquisition computing device and includes displaying the feedback and the revised feedback in a graphical user interface,( fig 5A, label 520 optionally automatically determine likely room corners and/ or borders, and display GUI that includes a first portion with the image and a user-manipulatable floor-wall, ceiling-wall and wall-wall room outline and optionally indications of likely corners and/or borders and that includes a second portion with an associated initial room shape) and wherein the stored contents include software instructions that, when executed by the image acquisition computing device, (para 54, In the illustrated embodiment, an embodiment of the ICA system 340 executes in memory 330 of the server computing system(s) 300 in order to perform at least some of the described techniques, such as by using the processor(s) 305 to execute software instructions of the system 340 in a manner that configures the processor(s) 305 and computing system 300 to perform automated operations that implement those described techniques).
The combined model does not explicitly teach displaying the completed floor plan in the graphical user interface.
While Artiano teaches displaying the completed floor plan in the graphical user interface (para, 04, Generating the floor plan may include generating and displaying, on the graphical user interface, a navigable three-dimensional model of the structure based upon, at least in part, the scanning of the one or more portions of the structure)
Artiano is considered to be analogous to the claimed invention since it focuses on floor plan generation. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date, to combine Artiano teaching of displaying the floor plan on GUI on to the modified model.
The motivation would have been by displaying the floor plan to the user the labeled floor plan and/or rendered model may provide emergency services or other third parties with an objective reference system for communicating and identifying the location of various portions of a structure or building (Artiano, para 35)
Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over LI, YUGUANG(CA 3090629 A1), in the view of Pham, Nam Thanh, Sihyun Park, and Chun-Su Park. "Fast and efficient method for large-scale aerial image stitching." IEEE Access 9 (2021), further in the view of Williams, Don, and Peter D. Burns. "Image Stitching: Exploring Practices, Software, and Performance." Archiving Conference. Vol. 10. Society of Imaging Science and Technology, 2013 further in the view of CubiCasa FAQ & Manual, retrieved on March 26, 2019, from hllps://www.cubi.casa/faq/, 5 pages, further in the view of Fathi; Habib (US 11106911 B1).
As of 30, the modified model teaches all the limitations of claim 29, and Pham also teaches predicting, by the one or more computing devices and based at least in part on one or more assessed attributes of the one first image, an adjusted predicted amount of resources involved with the producing of the floor plan for the building that is less than the predicted amount of resources based at least in part on the one or more improvements ( Section III, If the input images are too dense or several images share the same overlapping region, the dense correspondence may lead to mis-registration [8]. Consequently, the stitching performance of densely overlapped input images becomes inadequate [20], [25]. To address this problem, we propose a fast adaptive selection algorithm to eliminate unnecessary input images that are densely overlapped with other images. Accordingly, feature matching and registration processes are only performed in the overlapping regions of the correlated images. Thus, in the registration process, the proposed method can considerably reduce the number of outliers) On figure 12, it also shows the amount of time it takes was less after the above improvement on the proposed method and Pham also teaches conducted extensive experiments to prove the effectiveness of the proposed method in reducing the processing time.
displaying, by the image acquisition computing device, the adjusted predicted amount of resources involved with the producing of the floor plan for the building. (figure 12, display as a graph , section 4, The overall processing time of the proposed method for stitching 100, 150, and 200 images of resolution 5472 _ 3648 is 67.1, 106.5, and 135.7 (seconds), respectively).
The combined model does not explicitly teach acquiring, by the image acquisition computing device after the providing of the feedback, a new image to replace the one first image that includes one or more improvements to correct the one first image
While Fathi teaches acquiring, by the image acquisition computing device after the providing of the feedback, a new image to replace the one first image that includes one or more improvements to correct the one first image (para 63, (63) Such identification of one or more problematic image acquisition aspects or characteristics that occurred in a first image acquisition event and the reduction or elimination of such problematic aspects in a second image acquisition event can be generated automatically by creating a new capture plan for the second (or subsequent) image acquisition event or by modification of the first capture plan to include corrections as indicated from failure analysis of the output of the first image acquisition event. In this regard, correction of the second capture plan can be provided substantially without user interaction).
Fathi is considered to be analogous to the claimed invention since it focus on Image Acquisition Planning Systems. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date to integrate Fathi teaching of acquiring a new image to replace the problematic image on the modified model of floor plan generation process.
The motivation would have been capturing a second image to replace problematic images improve the accuracy and image acquisition process, it could be expected that the 3D information derivable from an image acquisition event be improved over time (Faith [62] –[66]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Childs; Bradley McKay (US 11314905 B2, Date Published, 2022-04-26) this invention is also analogous to the claimed invention since it focus on using a computing device to allow a user to quickly and conveniently take measurements of interior building features, and to create computerized floor plans of such features from any location within a space, without requiring the user to stay in a single location while taking the measurements
Dawson; Mitchell David (US 10708507 B1 Date Published 2020-07-07) this invention is also analogous to the claimed invention since it teaches automated operations to control acquisition of images in a defined area, including obtaining and using data from one or more hardware sensors.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABRHAM A. TAMIRU whose telephone number is (571)272-6987. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571 272 4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABRHAM ALEHEGN TAMIRU/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights